WinRT C++ (Win10) 从 SoftwareBitmap / BitmapBuffer 访问字节

Posted 2023-02-19

【中文标题】WinRT C++ (Win10) 从 SoftwareBitmap / BitmapBuffer 访问字节

【英文标题】：WinRT C++ (Win10) Accessing bytes from SoftwareBitmap / BitmapBuffer

【发布时间】：2015-10-28 18:38:57

【问题描述】：

要在 OpenCV 中处理我的相机 previewFrames，我需要访问原始像素数据/字节。所以，有新的 SoftwareBitmap，它应该正好提供这个。

c# 有一个 example，但在 Visual c++ 中我无法获得 IMemoryBufferByteAccess（见备注）接口工作。

异常代码：

// Capture the preview frame
return create_task(_mediaCapture->GetPreviewFrameAsync(videoFrame))
    .then([this](VideoFrame^ currentFrame)

    // Collect the resulting frame
    auto previewFrame = currentFrame->SoftwareBitmap;

    auto buffer = previewFrame->LockBuffer(Windows::Graphics::Imaging::BitmapBufferAccessMode::ReadWrite);
    auto reference = buffer->CreateReference();

    // Get a pointer to the pixel buffer
    byte* pData = nullptr;
    UINT capacity = 0;

    // Obtain ByteAccess
    ComPtr<IUnknown> inspectable = reinterpret_cast<IUnknown*>(buffer);
    // Query the IBufferByteAccess interface.
    Microsoft::WRL::ComPtr<IMemoryBufferByteAccess> bufferByteAccess;
    ThrowIfFailed(inspectable.As(&bufferByteAccess));     // ERROR ---> Throws HRESULT = E_NOINTERFACE

    // Retrieve the buffer data.
    ThrowIfFailed(bufferByteAccess->GetBuffer(_Out_ &pData, _Out_ &capacity));      // ERROR ---> Throws HRESULT = E_NOINTERFACE, because bufferByteAccess is null

我也试过这个：

    HRESULT hr = ((IMemoryBufferByteAccess*)reference)->GetBuffer(&pData, &capacity);

HRESULT 没问题，但我无法访问 pData -> 读取内存时访问冲突。

感谢您的帮助。

【问题讨论】：

看起来你应该使用Microsoft::WRL::ComPtr<IMemoryBufferByteAccess> 和reference，而不是buffer。

嗨。您是否设法在 HSV 颜色空间中显示图像？我有一些工件......在c＃（UWP）中，我将SoftwareBitmap传递给c ++，将SoftwareBitmap转换为cv::Mat，将颜色cpase更改为HSV，将Mat转换为SoftwareBitmap并传回c#..并且设置 Image.Source ... 而我所拥有的 .. onedrive.live.com/…

更详细，这里***.com/questions/34198259/…

【参考方案1】：

您应该在 reinterpret_cast 中使用引用而不是缓冲区。

#include "pch.h"
#include <wrl\wrappers\corewrappers.h>
#include <wrl\client.h>

MIDL_INTERFACE("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d") 
IMemoryBufferByteAccess : IUnknown

    virtual HRESULT STDMETHODCALLTYPE GetBuffer(
        BYTE   **value,
        UINT32 *capacity
        );
;

        auto previewFrame = currentFrame->SoftwareBitmap;

        auto buffer = previewFrame->LockBuffer(BitmapBufferAccessMode::ReadWrite);

        auto reference = buffer->CreateReference();

        ComPtr<IMemoryBufferByteAccess> bufferByteAccess;

        HRESULT result = reinterpret_cast<IInspectable*>(reference)->QueryInterface(IID_PPV_ARGS(&bufferByteAccess));

        if (result == S_OK)
        
            WriteLine("Get interface successfully");

            BYTE* data = nullptr;

            UINT32 capacity = 0;

            result = bufferByteAccess->GetBuffer(&data, &capacity);

            if (result == S_OK)
            
                WriteLine("get data access successfully, capacity: " + capacity);
            
        

【讨论】：

谢谢！ 3天的研究，很简单....你的代码甚至解决了我的内存分配问题

你能看看我的问题吗？ ***.com/questions/34198259/…

【参考方案2】：

我目前正在从我在MediaFrameReader::FrameArrived 事件中获得的每一帧中访问原始unsigned char* data，而不使用 WRL 和 COM...

方法如下：

void MainPage::OnFrameArrived(MediaFrameReader ^reader, MediaFrameArrivedEventArgs ^args)

    MediaFrameReference ^mfr = reader->TryAcquireLatestFrame();
    VideoMediaFrame ^vmf = mfr->VideoMediaFrame;
    VideoFrame ^vf = vmf->GetVideoFrame();
    SoftwareBitmap ^sb = vf->SoftwareBitmap; 
    Buffer ^buff = ref new Buffer(sb->PixelHeight * sb->PixelWidth * 2);
    sb->CopyToBuffer(buff);
    DataReader ^dataReader = DataReader::FromBuffer(buffer);
    Platform::Array<unsigned char, 1> ^arr = ref new Platform::Array<unsigned char, 1>(buffer->Length);
    dataReader->ReadBytes(arr);
    // here arr->Data is a pointer to the raw pixel data

注意：MediaCapture 对象需要配置MediaCaptureMemoryPreference::Cpu 才能拥有有效的SoftwareBitmap

希望以上内容对某人有所帮助

【参考方案3】：

基于来自@jeffrey-chen 的answer 和来自@kennykerr 的example，我组装了一个更清洁的解决方案：

#include <wrl/client.h>
// other includes, as required by your project

MIDL_INTERFACE("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")
IMemoryBufferByteAccess : ::IUnknown

    virtual HRESULT __stdcall GetBuffer(BYTE **value, UINT32 *capacity) = 0;
;

// your code:
    auto previewFrame = currentFrame->SoftwareBitmap;

    auto buffer = previewFrame->LockBuffer(BitmapBufferAccessMode::ReadWrite);

    auto bufferByteAccess= buffer->CreateReference().as<IMemoryBufferByteAccess>();
    WriteLine("Get interface successfully"); // otherwise - exception is thrown

    BYTE* data = nullptr;
    UINT32 capacity = 0;

    winrt::check_hresult(bufferByteAccess->GetBuffer(&data, &capacity));
    WriteLine("get data access successfully, capacity: " + capacity);