FFmpeg之Intel平台使用硬件加速

Posted 努力加贝

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了FFmpeg之Intel平台使用硬件加速相关的知识,希望对你有一定的参考价值。

FFmpeg之Intel平台使用硬件加速

着重讲解在不支持 QSV 的 Intel GPU 平台中如何使用 VAAPI 进行硬件解码


Quick Sync Video

Intel Quick Sync Video (QSV) 是 Intel GPU 上跟视频处理有关的一系列硬件特性的称呼

英特尔® Quick Sync Video 技术可以快速转换便携式多媒体播放器的视频,还能提供在线共享、视频编辑及视频制作功能,带有 Quick Sync Video 标识就表示该 CPU 支持硬件加速的视频编解码。

VAAPI Driver

Intel 提供了两种开源的 VAAPI 驱动

  • intel-vaapi-driver(i915/i965)
    • 在不支持 iHD 平台上建议尝试
  • intel-media-driver (iHD)
    • 优先推荐 iHD(QSV)
  • Supported Platforms(iHD)
    • BDW (Broadwell)
    • SKL (Skylake)
    • BXT (Broxton) / APL (Apollo Lake)
    • KBLx (KBL/Kaby Lake; CFL/Coffe Lake; WHL/Whiskey Lake; CML/Comet Lake; AML/Amber Lake)
    • ICL (Ice Lake)
    • JSL (Jasper Lake)/EHL (Elkhart Lake)
    • TGL (Tiger Lake)
    • DG1/SG1

Intel Media SDK(https://github.com/Intel-Media-SDK/MediaSDK)
intel-vaapi-driver(https://github.com/intel/intel-vaapi-driver)
intel-media-driver(https://github.com/intel/media-driver)
VAAPI(http://trac.ffmpeg.org/wiki/Hardware/VAAPI)
QuickSync(http://trac.ffmpeg.org/wiki/Hardware/QuickSync)


1)intel-media-driver 的使用
// 编译安装 iHD 驱动
https://github.com/intel/media-driver
// 编译 FFmpeg 开启 QSV 
sudo apt-get install libva-dev libmfx-dev intel-media-va-driver-non-free vainfo
./configure --arch=x86_64 --disable-yasm --enable-vaapi --enable-libmfx
make 
// 查看编译后的支持情况 ...
danoo@nulijiabei:~$ ffmpeg -decoders |grep h264
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.2.1-9ubuntu2)
  configuration: --arch=x86_64 --disable-yasm --enable-vaapi --enable-libmfx --disable-static --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
 VFS..D h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
 V....D h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
// 在系统中指定使用 iHD 驱动(Linux)
export LIBVA_DRIVER_NAME=iHD
// 通过解码器名称获取解码器
avcodec_find_decoder_by_name("h264_qsv");
2)intel-vaapi-driver 的使用
// 编译安装 i915/i965 驱动
https://github.com/intel/intel-vaapi-driver
// FFmepg 编译开启 VAAPI ...
apt-get install libva-dev
apt-get install build-essential
apt-get install pkg-config
./configure --prefix=/usr/local 
--enable-version3 
--disable-static 
--enable-shared 
--enable-vaapi 
--disable-x86asm
3)检查驱动是否正常
// 使用 vainfo 检查
$ vainfo
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.39 (libva 1.7.3)
vainfo: Driver version: Intel i965 driver for Intel(R) Skylake - 1.7.3
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
      VAProfileH264StereoHigh         : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointEncSlice
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
4)如何在 FFmpeg 中使用 VAAPI 解码

官方案例(https://github.com/FFmpeg/FFmpeg/blob/release/4.1/doc/examples/hw_decode.c)

/*
 * Copyright (c) 2017 Jun Zhao
 * Copyright (c) 2017 Kaixuan Liu
 *
 * HW Acceleration API (video decoding) decode sample
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */

/**
 * @file
 * HW-Accelerated decoding example.
 *
 * @example hw_decode.c
 * This example shows how to do HW-accelerated decoding with output
 * frames from the HW video surfaces.
 */

#include <stdio.h>

#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavutil/pixdesc.h>
#include <libavutil/hwcontext.h>
#include <libavutil/opt.h>
#include <libavutil/avassert.h>
#include <libavutil/imgutils.h>

static AVBufferRef *hw_device_ctx = NULL;
static enum AVPixelFormat hw_pix_fmt;
static FILE *output_file = NULL;

static int hw_decoder_init(AVCodecContext *ctx, const enum AVHWDeviceType type)
{
    int err = 0;

    if ((err = av_hwdevice_ctx_create(&hw_device_ctx, type,
                      NULL, NULL, 0)) < 0) {
    fprintf(stderr, "Failed to create specified HW device.\\n");
    return err;
    }
    ctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);

    return err;
}

static enum AVPixelFormat get_hw_format(AVCodecContext *ctx,
                    const enum AVPixelFormat *pix_fmts)
{
    const enum AVPixelFormat *p;

    for (p = pix_fmts; *p != -1; p++) {
    if (*p == hw_pix_fmt)
        return *p;
    }

    fprintf(stderr, "Failed to get HW surface format.\\n");
    return AV_PIX_FMT_NONE;
}

static int decode_write(AVCodecContext *avctx, AVPacket *packet)
{
    AVFrame *frame = NULL, *sw_frame = NULL;
    AVFrame *tmp_frame = NULL;
    uint8_t *buffer = NULL;
    int size;
    int ret = 0;

    ret = avcodec_send_packet(avctx, packet);
    if (ret < 0) {
    fprintf(stderr, "Error during decoding\\n");
    return ret;
    }

    while (1) {
    if (!(frame = av_frame_alloc()) || !(sw_frame = av_frame_alloc())) {
        fprintf(stderr, "Can not alloc frame\\n");
        ret = AVERROR(ENOMEM);
        goto fail;
    }

    ret = avcodec_receive_frame(avctx, frame);
    if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
        av_frame_free(&frame);
        av_frame_free(&sw_frame);
        return 0;
    } else if (ret < 0) {
        fprintf(stderr, "Error while decoding\\n");
        goto fail;
    }

    if (frame->format == hw_pix_fmt) {
        /* retrieve data from GPU to CPU */
        if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {
        fprintf(stderr, "Error transferring the data to system memory\\n");
        goto fail;
        }
        tmp_frame = sw_frame;
    } else
        tmp_frame = frame;

    size = av_image_get_buffer_size(tmp_frame->format, tmp_frame->width,
                    tmp_frame->height, 1);
    buffer = av_malloc(size);
    if (!buffer) {
        fprintf(stderr, "Can not alloc buffer\\n");
        ret = AVERROR(ENOMEM);
        goto fail;
    }
    ret = av_image_copy_to_buffer(buffer, size,
                      (const uint8_t * const *)tmp_frame->data,
                      (const int *)tmp_frame->linesize, tmp_frame->format,
                      tmp_frame->width, tmp_frame->height, 1);
    if (ret < 0) {
        fprintf(stderr, "Can not copy image to buffer\\n");
        goto fail;
    }

    if ((ret = fwrite(buffer, 1, size, output_file)) < 0) {
        fprintf(stderr, "Failed to dump raw data.\\n");
        goto fail;
    }

    fail:
    av_frame_free(&frame);
    av_frame_free(&sw_frame);
    av_freep(&buffer);
    if (ret < 0)
        return ret;
    }
}

int main(int argc, char *argv[])
{
    AVFormatContext *input_ctx = NULL;
    int video_stream, ret;
    AVStream *video = NULL;
    AVCodecContext *decoder_ctx = NULL;
    AVCodec *decoder = NULL;
    AVPacket packet;
    enum AVHWDeviceType type;
    int i;

    if (argc < 4) {
    fprintf(stderr, "Usage: %s <device type> <input file> <output file>\\n", argv[0]);
    return -1;
    }

    type = av_hwdevice_find_type_by_name(argv[1]);
    if (type == AV_HWDEVICE_TYPE_NONE) {
    fprintf(stderr, "Device type %s is not supported.\\n", argv[1]);
    fprintf(stderr, "Available device types:");
    while((type = av_hwdevice_iterate_types(type)) != AV_HWDEVICE_TYPE_NONE)
        fprintf(stderr, " %s", av_hwdevice_get_type_name(type));
    fprintf(stderr, "\\n");
    return -1;
    }

    /* open the input file */
    if (avformat_open_input(&input_ctx, argv[2], NULL, NULL) != 0) {
    fprintf(stderr, "Cannot open input file '%s'\\n", argv[2]);
    return -1;
    }

    if (avformat_find_stream_info(input_ctx, NULL) < 0) {
    fprintf(stderr, "Cannot find input stream information.\\n");
    return -1;
    }

    /* find the video stream information */
    ret = av_find_best_stream(input_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &decoder, 0);
    if (ret < 0) {
    fprintf(stderr, "Cannot find a video stream in the input file\\n");
    return -1;
    }
    video_stream = ret;

    for (i = 0;; i++) {
    const AVCodecHWConfig *config = avcodec_get_hw_config(decoder, i);
    if (!config) {
        fprintf(stderr, "Decoder %s does not support device type %s.\\n",
            decoder->name, av_hwdevice_get_type_name(type));
        return -1;
    }
    if (config->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX &&
        config->device_type == type) {
        hw_pix_fmt = config->pix_fmt;
        break;
    }
    }

    if (!(decoder_ctx = avcodec_alloc_context3(decoder)))
    return AVERROR(ENOMEM);

    video = input_ctx->streams[video_stream];
    if (avcodec_parameters_to_context(decoder_ctx, video->codecpar) < 0)
    return -1;

    decoder_ctx->get_format  = get_hw_format;

    if (hw_decoder_init(decoder_ctx, type) < 0)
    return -1;

    if ((ret = avcodec_open2(decoder_ctx, decoder, NULL)) < 0) {
    fprintf(stderr, "Failed to open codec for stream #%u\\n", video_stream);
    return -1;
    }

    /* open the file to dump raw data */
    output_file = fopen(argv[3], "w+");

    /* actual decoding and dump the raw data */
    while (ret >= 0) {
    if ((ret = av_read_frame(input_ctx, &packet)) < 0)
        break;

    if (video_stream == packet.stream_index)
        ret = decode_write(decoder_ctx, &packet);

    av_packet_unref(&packet);
    }

    /* flush the decoder */
    packet.data = NULL;
    packet.size = 0;
    ret = decode_write(decoder_ctx, &packet);
    av_packet_unref(&packet);

    if (output_file)
    fclose(output_file);
    avcodec_free_context(&decoder_ctx);
    avformat_close_input(&input_ctx);
    av_buffer_unref(&hw_device_ctx);

    return 0;
}
// 相关文档 ...
https://www.jianshu.com/p/3ea9ef713211
https://my.oschina.net/u/4416603/blog/3239100/print

// Golang 封装的 FFmepg
https://github.com/ailumiyana/streaming/blob/master/vaapi_qsv_test/qsv-test.go

以上是关于FFmpeg之Intel平台使用硬件加速的主要内容,如果未能解决你的问题,请参考以下文章

Android开发之《硬件加速》

FFmpeg使用显卡进行转码硬件加速的记录,以及和软压的比较

FFmpeg 硬件加速方案概览 (下)

几个平台环境里视频编解码和图像scale的硬件加速的方法

Linux 下的模拟器硬件加速

ffmpeg使用硬件加速hwaccelcuvidh264_cuvidh264_nvenc