将 AVFrame 转换为 RGB32 时在 sws_scale 处崩溃
Posted
技术标签:
【中文标题】将 AVFrame 转换为 RGB32 时在 sws_scale 处崩溃【英文标题】：Crash at sws_scale when converting AVFrame to RGB32 【发布时间】：2017-02-28 08:56:01 【问题描述】：转换 AVPicture 时在 sws_scale 处崩溃
起初我使用 sws_scale 来实际放大帧，但 cpu 开销太高，所以我决定只转换帧并调整 QImage 大小。在它工作之前，我在渲染时得到了视频显示，但现在它在 sws_scale 处崩溃。
这是用 Qt for android 编写的，使用 FFMpeg 3.1.4。
另外，还有其他方法可以不使用已弃用的函数吗？
有人知道我为什么会在 sws_scale 崩溃吗？
VideoFrameCopy 的类
class VideoFrameCopy
public:
VideoFrameCopy()
VideoFrameCopy(AVFrame *frame) copyAVFrame(frame);
~VideoFrameCopy();
void copyAVFrame(AVFrame *frame); // copy essential data from AVFrame
AVPicture picture;
int64_t pkt_pts = -1; // show it hasn't been initialised
int64_t best_pts;
int interlaced_frame;
int width = 0, height = 0;
int format = -1;
;
将帧转换为RGBA8888 QImage的代码
if (frame)
if (image->width() != vid_ctx->width || image->height() != vid_ctx->height)
QSize old_size(image->size());
// block until renderer has finished with it
while (parent->buffer_ready)
QThread::yieldCurrentThread();
delete image;
image = new QImage(vid_ctx->width, vid_ctx->height, QImage::Format_RGBA8888);
parent->image = image;
if (scale_context) sws_freeContext(scale_context);
scale_context = nullptr;
qDebug() << "Video image size" << image->size() << "old" << old_size;
// the src width and height may need to change to use the context info instead
if (!scale_context) // create the scale context
int src_width = vid_ctx->width;
int src_height = vid_ctx->height;
AVPixelFormat src_format = vid_ctx->pix_fmt;//(AVPixelFormat)frame->format;
int dst_width = vid_ctx->width;
int dst_height = vid_ctx->height;
AVPixelFormat dst_format = AV_PIX_FMT_RGBA;
scale_context = sws_getContext(src_width, src_height, src_format,
dst_width, dst_height, dst_format,
SWS_FAST_BILINEAR, NULL, NULL, NULL);
av_image_fill_linesizes(scale_linesizes, dst_format, vid_ctx->width);
qDebug() << "Created scale context" << scale_context;
if (scale_context) // valid
scale_data[0] = image->bits();
sws_scale(scale_context,
frame->picture.data, // deprecated
frame->picture.linesize, // deprecated
0, image->height(),
scale_data,
scale_linesizes);
qDebug() << "Frame converted";
//av_frame_unref(frame);
//vid_frames_mutex.lock();
//if (quit) av_frame_free(&frame);
if (quit) delete frame;
else vid_frames_unused.push_back(frame);
//vid_frames_mutex.unlock();
//qDebug() << "got frame" << clock_current_frame_last << "clock" << clock_current_time;
vid_frames_mutex.unlock();
return frame != nullptr;
VideoFrameCopy 类的函数
void VideoFrameCopy::copyAVFrame(AVFrame *frame)
if (pkt_pts != -1 &&
(width != frame->width ||
height != frame->height ||
format != frame->format)
) // picture changed?
avpicture_free(&picture); // deprecated
pkt_pts = -1;
width = frame->width;
height = frame->height;
format = frame->format;
interlaced_frame = frame->interlaced_frame;
if (pkt_pts == -1) // alloc picture
if (avpicture_alloc(&picture, (AVPixelFormat)format, width, height) < 0) return; // deprecated
int size = avpicture_get_size((AVPixelFormat)format, width, height); // deprecated
uint8_t *picture_data = (uint8_t*)av_malloc(size);
avpicture_fill(&picture, picture_data, (AVPixelFormat)format, width, height); // deprecated
qDebug() << "New frame" << width << "x" << height << format;
pkt_pts = frame->pkt_pts;
best_pts = av_frame_get_best_effort_timestamp(frame);
av_picture_copy(&picture, (AVPicture*)frame, (AVPixelFormat)format, width, height); // deprecated
qDebug() << "picture" << picture.linesize[0] << picture.linesize[1]; // deprecated
VideoFrameCopy::~VideoFrameCopy()
if (pkt_pts != -1)
/*if (picture.data)
av_free(picture.data);
picture.data = nullptr;
*/
avpicture_free(&picture); // deprecated
logcat 的示例输出
D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:659 (void VideoFrameCopy::copyAVFrame(AVFrame*)): New frame 640 x 358 0
D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:667 (void VideoFrameCopy::copyAVFrame(AVFrame*)): picture 640 320
D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:586 (bool FFMpegFile::getVideoFrame()): Video image size QSize(640, 358) old QSize(500, 320)
D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:659 (void VideoFrameCopy::copyAVFrame(AVFrame*)): New frame 640 x 358 0
D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:606 (bool FFMpegFile::getVideoFrame()): Created scale context 0x4bb49060
D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:667 (void VideoFrameCopy::copyAVFrame(AVFrame*)): picture 640 320
F/libc (20157): Fatal signal 7 (SIGBUS) at 0x4e065008 (code=1), thread 20335 (QThread)
I/DEBUG ( 116): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG ( 116): Build fingerprint: 'ODROID/odroidc/odroidc:4.4.2/KOT49H/odroidc-eng-s805_4.4.2_master-410:eng/test-keys'
I/DEBUG ( 116): Revision: '10'
I/DEBUG ( 116): pid: 20157, tid: 20335, name: QThread >>> org.qtproject.example <<<
I/DEBUG ( 116): signal 7 (SIGBUS), code 1 (BUS_ADRALN), fault addr 4e065008
I/DEBUG ( 116): r0 00000280 r1 00000166 r2 4e065008 r3 00000a00
I/DEBUG ( 116): r4 4d9bd030 r5 00000280 r6 4d9f4f28 r7 00000140
I/DEBUG ( 116): r8 00000280 r9 00000010 sl 4da02ee8 fp 4e065a08
I/DEBUG ( 116): ip 4d9bd2a0 sp 4c1948f8 lr 4a8f1d2c pc 4a8f4dc0 cpsr 280f0010
I/DEBUG ( 116): d0 004a004a004a004a d1 0081ffccffe70066
I/DEBUG ( 116): d2 004a004a004a004a d3 0000000000000000
I/DEBUG ( 116): d4 0000000000000000 d5 0000000000000000
I/DEBUG ( 116): d6 0000000001010101 d7 0000000001010101
I/DEBUG ( 116): d8 0000000001010101 d9 ffffffffffffffff
I/DEBUG ( 116): d10 0000000000000000 d11 0000000000000000
I/DEBUG ( 116): d12 0000000000000000 d13 ffffffffffffffff
I/DEBUG ( 116): d14 004a004a004a004a d15 004a004a004a004a
I/DEBUG ( 116): d16 0000000000000000 d17 0000000000000000
I/DEBUG ( 116): d18 0000000000000000 d19 0000000000000000
I/DEBUG ( 116): d20 0000000000000000 d21 0000000000000000
I/DEBUG ( 116): d22 0000000000000000 d23 0000000000000000
I/DEBUG ( 116): d24 0000000000000000 d25 0000000000000000
I/DEBUG ( 116): d26 0000000000000000 d27 0000000000000000
I/DEBUG ( 116): d28 004a004a004a004a d29 0000000000000000
I/DEBUG ( 116): d30 0000000000000000 d31 0000000000000000
I/DEBUG ( 116): scr 20000010
I/DEBUG ( 116):
I/DEBUG ( 116): backtrace:
I/DEBUG ( 116): #00 pc 0000edc0 /data/app-lib/org.qtproject.example-1/libswscale-4.so
I/DEBUG ( 116): #01 pc 0000bd28 /data/app-lib/org.qtproject.example-1/libswscale-4.so
I/DEBUG ( 116):
I/DEBUG ( 116): stack:
I/DEBUG ( 116): 4c1948b8 0000004b
I/DEBUG ( 116): 4c1948bc 4bb2d270
I/DEBUG ( 116): 4c1948c0 0000013c
I/DEBUG ( 116): 4c1948c4 4011edbc /system/lib/libc.so (dlmalloc+480)
I/DEBUG ( 116): 4c1948c8 4c19494a [stack:20335]
I/DEBUG ( 116): 4c1948cc 4015e384
I/DEBUG ( 116): 4c1948d0 00000010
I/DEBUG ( 116): 4c1948d4 489033ef /data/app-lib/org.qtproject.example-1/libQt5Core.so
I/DEBUG ( 116): 4c1948d8 00001000
I/DEBUG ( 116): 4c1948dc 00000000
I/DEBUG ( 116): 4c1948e0 4bb2d470
I/DEBUG ( 116): 4c1948e4 4bb2d478
I/DEBUG ( 116): 4c1948e8 4bb2d478
I/DEBUG ( 116): 4c1948ec 4bb2d470
I/DEBUG ( 116): 4c1948f0 00000002
I/DEBUG ( 116): 4c1948f4 4012109c /system/lib/libc.so (dlfree+996)
I/DEBUG ( 116): #00 4c1948f8 11111111
I/DEBUG ( 116): ........ ........
I/DEBUG ( 116): #01 4c1948f8 11111111
I/DEBUG ( 116): 4c1948fc 3fa11111
I/DEBUG ( 116): 4c194900 40000000
I/DEBUG ( 116): 4c194904 40640d79 /system/lib/libskia.so
I/DEBUG ( 116): 4c194908 00000000
I/DEBUG ( 116): 4c19490c 3ff00000
I/DEBUG ( 116): 4c194910 00000000
I/DEBUG ( 116): 4c194914 3ff00000
I/DEBUG ( 116): 4c194918 00000000
I/DEBUG ( 116): 4c19491c 3ff00000
I/DEBUG ( 116): 4c194920 00000000
I/DEBUG ( 116): 4c194924 3f800000
I/DEBUG ( 116): 4c194928 00000000
I/DEBUG ( 116): 4c19492c 00000000
I/DEBUG ( 116): 4c194930 00000000
I/DEBUG ( 116): 4c194934 00000000
I/DEBUG ( 116):
I/DEBUG ( 116): memory near r2:
I/DEBUG ( 116): 4e064fe8 00000000 00000000 00000000 00000007
...
I/DEBUG ( 116): memory map around fault addr 4e065008:
I/DEBUG ( 116): 4dd15000-4df15000 rw- /dev/mali
I/DEBUG ( 116): 4df15000-4e199000 rw-
I/DEBUG ( 116): 4e676000-4e876000 rw- /dev/mali
I/BootReceiver( 479): Copying /data/tombstones/tombstone_07 to DropBox (SYSTEM_TOMBSTONE)
W/ActivityManager( 479): Force finishing activity org.qtproject.example/org.qtproject.qt5.android.bindings.QtActivity
I/WindowState( 479): WIN DEATH: Window64cf3a40 u0 org.qtproject.example/org.qtproject.qt5.android.bindings.QtActivity
I/WindowState( 479): WIN DEATH: Window64d0f6e0 u0 SurfaceView
I/UsageStats( 479): No package stats for pkg:org.qtproject.example
I/art ( 118): Process 20157 terminated by signal (7)
W/ActivityManager( 479): Exception thrown during pause
W/ActivityManager( 479): android.os.DeadObjectException
W/ActivityManager( 479): at android.os.BinderProxy.transact(Native Method)
W/ActivityManager( 479): at android.app.ApplicationThreadProxy.schedulePauseActivity(ApplicationThreadNative.java:660)
W/ActivityManager( 479): at com.android.server.am.ActivityStack.startPausingLocked(ActivityStack.java:778)
W/ActivityManager( 479): at com.android.server.am.ActivityStack.finishActivityLocked(ActivityStack.java:2614)
W/ActivityManager( 479): at com.android.server.am.ActivityStack.finishTopRunningActivityLocked(ActivityStack.java:2488)
W/ActivityManager( 479): at com.android.server.am.ActivityStackSupervisor.finishTopRunningActivityLocked(ActivityStackSupervisor.java:2196)
W/ActivityManager( 479): at com.android.server.am.ActivityManagerService.handleAppCrashLocked(ActivityManagerService.java:9705)
W/ActivityManager( 479): at com.android.server.am.ActivityManagerService.makeAppCrashingLocked(ActivityManagerService.java:9598)
W/ActivityManager( 479): at com.android.server.am.ActivityManagerService.crashApplication(ActivityManagerService.java:10243)
W/ActivityManager( 479): at com.android.server.am.ActivityManagerService.handleApplicationCrashInner(ActivityManagerService.java:9794)
W/ActivityManager( 479): at com.android.server.am.NativeCrashListener$NativeCrashReporter.run(NativeCrashListener.java:86)
D/ActivityManager( 479): resumeClassName is com.android.launcher2.Launcher
D/ActivityManager( 479): resumePackageName is com.android.launcher
I/ActivityManager( 479): Process org.qtproject.example (pid 20157) has died.
D/ActivityManager( 479): send app_CRASH broadcast, packageName:org.qtproject.example
如果有人能提供帮助，不胜感激......
【问题讨论】：
【参考方案1】：关于崩溃：了解访问冲突是否与读取输入图片或写入输出图片有关会有所帮助。无论如何，到目前为止，我只能看到代码的一个问题：当使用 QImage 作为目标时，您不能使用av_image_fill_linesizes。 Linesizes 必须匹配实际的图像布局（就像数据指针必须匹配内存中的实际图像位置）所以使用这样的东西：
scale_linesizes[0] = image->bytesPerLine();
关于所有已弃用的功能：您永远不需要使用 AVPicture。 AVFrame 可以做 AVPicture 可以做的所有事情，然后是一些。例如。如果你有一个“普通”的 AVFrame（例如来自 FFmpeg 的解码器）并且你想保留一个副本，请使用 av_frame_ref 或 av_frame_copy。这将分配内存并复制图片，或者如果可能的话，它将仅共享图片缓冲区。如果您有自己填写的 AVFrame，那么相同的函数调用应该是一样的（假设您没有填写 frame->buf[] 数组）。如果您有一个框架并且想要获得自己的“私人”/非共享副本，您可以使用av_frame_make_writable，它会制作一个副本（如果需要）。最后，如果你想无条件地复制图片数据（你不应该这样做，但如果出于某种原因你需要），然后做av_frame_alloc，av_frame_copy_props，填充宽度/高度/格式字段，av_frame_get_buffer，最后av_frame_copy.
【讨论】：
是的。非常感谢您的解释。这解决了我遇到的很多问题，尤其是复制图片数据的顺序。我对代码进行了一些重大更改，主要是取消缩放并将图像转换为 RGBA，并使用未弃用的 API 调用。这个答案将是我的参考。 关于崩溃，这仍然是我不确定的事情，尽管这可能是 AVFrame 中的错误设置，并且正如你所说，设置帧副本..以上是关于将 AVFrame 转换为 RGB32 时在 sws_scale 处崩溃的主要内容，如果未能解决你的问题，请参考以下文章
如何从AVFrame：data里获取RGB24数据和YUYV422数据