内存泄漏（OpenCV + QML）

Posted 2023-02-14

【中文标题】内存泄漏（OpenCV + QML）

【英文标题】：Memory leak (OpenCV + QML)

【发布时间】：2017-11-02 09:32:47

【问题描述】：

我正在尝试在 QML 图像中显示 OpenCV Mat。

我使用 OpenCV 从相机中抓取帧，帧在 QML 中成功显示，但内存使用量随时间增加。我该如何解决？这是我的代码：

main.cpp

#include <QGuiApplication>
#include "videoprovider.h"

int main(int argc, char *argv[])

    QGuiApplication app(argc, argv);
    VideoProvider videoProvider;
    return app.exec();

VideoProvider.h

#ifndef VIDEOPROVIDER_H
#define VIDEOPROVIDER_H

#include <QObject>
#include <QFuture>
#include <QImage>
#include <QQmlApplicationEngine>
#include <QQuickImageProvider>
#include <opencv2/opencv.hpp>

class VideoProvider : public QObject, public QQuickImageProvider

    Q_OBJECT
public:
    explicit VideoProvider();
    QPixmap requestPixmap(const QString &id, QSize *size, const QSize& requestedSize);

signals:
    void frameChanged();

public slots:
    void framePainted();

private:
    QQmlApplicationEngine engine;
    bool readyfor;
    cv::Mat mat;
    QImage outputImage;
    void process();
;

#endif // VIDEOPROVIDER_H

VideoProvider.cpp

#include <QQmlContext>
#include <QtConcurrent/QtConcurrent>
#include <QDebug>
#include <QThread>
#include "videoprovider.h"
#include <QQuickImageProvider>

VideoProvider::VideoProvider() : QQuickImageProvider (QQuickImageProvider :: Pixmap)

    engine.rootContext()->setContextProperty("videoProvider", this);
    engine.addImageProvider(QLatin1String ("videoCapture"), this);
    engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
    readyfor = true;
    QtConcurrent::run(this, VideoProvider::process);


void VideoProvider::framePainted()

    readyfor = true;


void VideoProvider::process()

    cv::VideoCapture capture(0);

    while(true)

        QThread::currentThread()->msleep(80);

        if(!readyfor) continue;

        mat.release();
        capture >> mat;

        if(mat.empty())
        
            qDebug()<<"disconnect";
        
        else
        
            readyfor = false;
            cv::cvtColor(mat, mat, cv::COLOR_BGR2RGB);
            outputImage = QImage((uchar*)mat.data, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
            emit frameChanged();
        
    

    capture.release();


QPixmap VideoProvider::requestPixmap(const QString &id, QSize *size, const QSize& requestedSize)

    return QPixmap::fromImage(outputImage);

main.qml

import QtQuick 2.6
import QtQuick.Window 2.2

Window

    visible: true
    width: 640
    height: 480
    title: qsTr("Hello World")
    id: root

    Image
        id: videoLayer
        anchors.fill: parent
        cache: false

        onSourceChanged:
            videoProvider.framePainted();
        
    

    Connections
    
        target: videoProvider
        property int frameCounter: 0

        onFrameChanged:
        
            videoLayer.source = "image://videoCapture/hoge" + frameCounter;
            frameCounter ^= 1;
        
    

我发现它发生在我向 QML 发送信号 (emit frameChanged();) 时。

UPD：

Valgrind 日志：

==18038== Memcheck, a memory error detector
==18038== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==18038== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==18038== Command: ./prog
==18038== 
QML debugging is enabled. Only use this in a safe environment.
==18038== Warning: noted but unhandled ioctl 0x30000001 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x27 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x7ff with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x25 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x17 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: set address range perms: large range [0x200000000, 0x500000000) (noaccess)
==18038== Warning: set address range perms: large range [0x500000000, 0x700000000) (noaccess)
==18038== Warning: noted but unhandled ioctl 0x19 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x21 with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Warning: noted but unhandled ioctl 0x1b with no size/direction hints.
==18038==    This could cause spurious value errors to appear.
==18038==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18038== Thread 8 Thread (pooled):
==18038== Invalid read of size 4
==18038==    at 0xAF682D0: QImage::~QImage() (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0x40570F: VideoProvider::process() (videoprovider.cpp:50)
==18038==    by 0x406AEB: QtConcurrent::VoidStoredMemberFunctionPointerCall0<void, VideoProvider>::runFunctor() (qtconcurrentstoredfunctioncall.h:205)
==18038==    by 0x405FC6: QtConcurrent::RunFunctionTask<void>::run() (qtconcurrentrunbase.h:136)
==18038==    by 0xBC44BA2: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xBC48849: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xCE456B9: start_thread (pthread_create.c:333)
==18038==    by 0xC9773DC: clone (clone.S:109)
==18038==  Address 0x23d1b260 is 0 bytes inside a block of size 128 free'd
==18038==    at 0x4C2F24B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18038==    by 0xAF682F3: QImage::~QImage() (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0x4047CB: VideoProvider::~VideoProvider() (videoprovider.h:11)
==18038==    by 0x403A34: main (main.cpp:8)
==18038==  Block was alloc'd at
==18038==    at 0x4C2E0EF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18038==    by 0xAF656BE: QImageData::create(unsigned char*, int, int, int, QImage::Format, bool, void (*)(void*), void*) (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0xAF65971: QImage::QImage(unsigned char*, int, int, int, QImage::Format, void (*)(void*), void*) (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Gui.so.5.9.1)
==18038==    by 0x4056E2: VideoProvider::process() (videoprovider.cpp:50)
==18038==    by 0x406AEB: QtConcurrent::VoidStoredMemberFunctionPointerCall0<void, VideoProvider>::runFunctor() (qtconcurrentstoredfunctioncall.h:205)
==18038==    by 0x405FC6: QtConcurrent::RunFunctionTask<void>::run() (qtconcurrentrunbase.h:136)
==18038==    by 0xBC44BA2: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xBC48849: ??? (in /home/dmytro/Qt/5.9.1/gcc_64/lib/libQt5Core.so.5.9.1)
==18038==    by 0xCE456B9: start_thread (pthread_create.c:333)
==18038==    by 0xC9773DC: clone (clone.S:109)

【问题讨论】：

您是否尝试对您的项目运行内存检查？尝试运行valgrind --leak-check=full your-app-path

只要 OpenCV 有自己的线程并且 UI 与它完全异步，我就会通过队列连接通过信号将图像发送到 UI。否则，您需要阻止对该相机图像 QImage 的访问。如果你不这样做，你也在寻找麻烦。在发送图像之前查看 QImage::bits 对深层副本的调用。

是的！第一次我的程序崩溃是因为我试图同时使用不同线程的QImage。然后我添加了布尔变量readyfor 来防止这种情况。现在我从 QML 发送信号，表示 Image 已准备好获取新帧，然后开始抓取新帧。

【参考方案1】：

我使用 OpenCV 从相机中抓取帧，显示帧 在 QML 中成功，但内存使用量随着时间的推移而增加。我怎样才能 修好吗？

以下前提条件是视频捕获处理图像的速度不超过每帧 1000 毫秒 / 80 毫秒 = 每秒 12.5 帧（实际上由于处理时间的原因而更少），而标准相机速率在 25 到 80 范围内。这就是为什么图像帧卡在 OpenCV 等的内部缓冲区中的原因。只要操作系统仍然能够为其他线程放弃时间片，在这里等待 1 到 5 毫秒（必须足够）不会有什么坏处。我个人使用来自C++ 11 或Qt 的条件变量，而不是“睡眠”。使用条件变量，我们可以更优雅地中断等待等。

while(true)

    QThread::currentThread()->msleep(80); // too much wait
    // also just do QThread::msleep instead

    if(!readyfor) continue;

    mat.release();
    capture >> mat

【讨论】：

感谢您的回复！真的 80 毫秒等待太多了。但似乎问题不在 OpenCV 的缓冲区中。我从我的项目和函数中删除了 OpenCV：QPixmap VideoProvider::requestPixmap(const QString &id, QSize *size, const QSize& requestedSize) return QPixmap::fromImage(outputImage); start return:return QPixmap(500, 500); 在 QML 中它看起来像黑色矩形，即使在这一点上，内存使用量也会增加。看起来，这些 QPixmap 被缓存并增加了内存使用量。而且我不知道如何清理内存。

您也可以尝试QQuickImageProvider::requestImage，我什至不确定您为什么要使用图像提供程序？我在一些应用程序中使用doc.qt.io/qt-5/qml-qtmultimedia-videooutput.html 并使用它的源表面对象：来自 C++ 代码的doc.qt.io/qt-5/qml-qtmultimedia-videooutput.html#source-prop。

谢谢！我也会尝试使用 VideoOutput。我什至没有想到使用这种方法！也许你有一个小例子如何正确地做到这一点？

有很多，例如：github.com/chili-epfl/qml-cvcamera

太棒了！非常感谢！