FFmpeg开发教程FFmpeg 版 Hello world

Posted 2021-11-28 ichenwin

本系列根据项目ffmpeg-libav-tutorial翻译而来

Chapter 0 - 万物之源 —— hello world

然而，本节的程序并不会在终端打印“Hello world”，而是会打印原视频文件的一些信息，比如封装格式、视频时长、分辨率、音频通道数。最后，我们将解码每一帧视频，并将它们保存为YUV文件。

FFmpeg libav 架构

在开始coding之前，我们先来了解下FFmpeg的代码结构，并了解它的各个组件之间是怎么通信的。

下图展示了视频解码的大致流程：

首先，我们需要将媒体文件读取出来，并保存至AVFormatContext 组件。(视频封装container，也叫视频格式format).
通常，FFmpeg并不会将整个视频文件加载进来，只是获取了视频的头部。

一旦我们拿到了封装头部，我们便可以访问它所包含的所有媒体流（一般包含视频流、音频流）。
每一路媒体流可以通过 AVStream进行访问。

Stream is a fancy name for a continuous flow of data.

Suppose our video has two streams: an audio encoded with AAC CODEC and a video encoded with H264 (AVC) CODEC. From each stream we can extract pieces (slices) of data called packets that will be loaded into components named AVPacket.

The data inside the packets are still coded (compressed) and in order to decode the packets, we need to pass them to a specific AVCodec.

The AVCodec will decode them into AVFrame and finally, this component gives us the uncompressed frame. Noticed that the same terminology/process is used either by audio and video stream.

Requirements

Since some people were facing issues while compiling or running the examples we‘re going to use Docker as our development/runner environment, we‘ll also use the big buck bunny video so if you don‘t have it locally just run the command make fetch_small_bunny_video.

Chapter 0 - code walkthrough

TLDR; show me the code and execution.

$ make run_hello

We‘ll skip some details, but don‘t worry: the source code is available at github.

We‘re going to allocate memory to the component AVFormatContext that will hold information about the format (container).

AVFormatContext *pFormatContext = avformat_alloc_context();

Now we‘re going to open the file and read its header and fill the AVFormatContext with minimal information about the format (notice that usually the codecs are not opened).
The function used to do this is avformat_open_input. It expects an AVFormatContext, a filename and two optional arguments: the AVInputFormat (if you pass NULL, FFmpeg will guess the format) and the AVDictionary (which are the options to the demuxer).

avformat_open_input(&pFormatContext, filename, NULL, NULL);

We can print the format name and the media duration:

printf("Format %s, duration %lld us", pFormatContext->iformat->long_name, pFormatContext->duration);

To access the streams, we need to read data from the media. The function avformat_find_stream_info does that.
Now, the pFormatContext->nb_streams will hold the amount of streams and the pFormatContext->streams[i] will give us the i stream (an AVStream).

avformat_find_stream_info(pFormatContext,  NULL);

Now we‘ll loop through all the streams.

for (int i = 0; i < pFormatContext->nb_streams; i++)
{
  //
}

For each stream, we‘re going to keep the AVCodecParameters, which describes the properties of a codec used by the stream i.

AVCodecParameters *pLocalCodecParameters = pFormatContext->streams[i]->codecpar;

With the codec properties we can look up the proper CODEC querying the function avcodec_find_decoder and find the registered decoder for the codec id and return an AVCodec, the component that knows how to enCOde and DECode the stream.

AVCodec *pLocalCodec = avcodec_find_decoder(pLocalCodecParameters->codec_id);

Now we can print information about the codecs.

// specific for video and audio
if (pLocalCodecParameters->codec_type == AVMEDIA_TYPE_VIDEO) {
  printf("Video Codec: resolution %d x %d", pLocalCodecParameters->width, pLocalCodecParameters->height);
} else if (pLocalCodecParameters->codec_type == AVMEDIA_TYPE_AUDIO) {
  printf("Audio Codec: %d channels, sample rate %d", pLocalCodecParameters->channels, pLocalCodecParameters->sample_rate);
}
// general
printf("\\tCodec %s ID %d bit_rate %lld", pLocalCodec->long_name, pLocalCodec->id, pCodecParameters->bit_rate);

With the codec, we can allocate memory for the AVCodecContext, which will hold the context for our decode/encode process, but then we need to fill this codec context with CODEC parameters; we do that with avcodec_parameters_to_context.

Once we filled the codec context, we need to open the codec. We call the function avcodec_open2 and then we can use it.

AVCodecContext *pCodecContext = avcodec_alloc_context3(pCodec);
avcodec_parameters_to_context(pCodecContext, pCodecParameters);
avcodec_open2(pCodecContext, pCodec, NULL);

Now we‘re going to read the packets from the stream and decode them into frames but first, we need to allocate memory for both components, the AVPacket and AVFrame.

AVPacket *pPacket = av_packet_alloc();
AVFrame *pFrame = av_frame_alloc();

Let‘s feed our packets from the streams with the function av_read_frame while it has packets.

while (av_read_frame(pFormatContext, pPacket) >= 0) {
  //...
}

Let‘s send the raw data packet (compressed frame) to the decoder, through the codec context, using the function avcodec_send_packet.

avcodec_send_packet(pCodecContext, pPacket);

And let‘s receive the raw data frame (uncompressed frame) from the decoder, through the same codec context, using the function avcodec_receive_frame.

avcodec_receive_frame(pCodecContext, pFrame);

We can print the frame number, the PTS, DTS, frame type and etc.

printf(
    "Frame %c (%d) pts %d dts %d key_frame %d [coded_picture_number %d, display_picture_number %d]",
    av_get_picture_type_char(pFrame->pict_type),
    pCodecContext->frame_number,
    pFrame->pts,
    pFrame->pkt_dts,
    pFrame->key_frame,
    pFrame->coded_picture_number,
    pFrame->display_picture_number
);

Finally we can save our decoded frame into a simple gray image. The process is very simple, we‘ll use the pFrame->data where the index is related to the planes Y, Cb and Cr, we just picked 0 (Y) to save our gray image.

save_gray_frame(pFrame->data[0], pFrame->linesize[0], pFrame->width, pFrame->height, frame_filename);

static void save_gray_frame(unsigned char *buf, int wrap, int xsize, int ysize, char *filename)
{
    FILE *f;
    int i;
    f = fopen(filename,"w");
    // writing the minimal required header for a pgm file format
    // portable graymap format -> https://en.wikipedia.org/wiki/Netpbm_format#PGM_example
    fprintf(f, "P5\\n%d %d\\n%d\\n", xsize, ysize, 255);

    // writing line by line
    for (i = 0; i < ysize; i++)
        fwrite(buf + i * wrap, 1, xsize, f);
    fclose(f);
}

And voilà! Now we have a gray scale image with 2MB:

最后贴上完整的代码，本教程所有代码也将存放在Github: FFmpegSimplePlayer项目中：

#include <stdio.h>
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

static void save_gray_frame(unsigned char *buf, int wrap, int xsize, int ysize, char *filename)
{
    FILE *f = NULL;
    int i = 0;
    f = fopen(filename, "w");
    fprintf(f, "P5\\n%d %d\\n%d\\n", xsize, ysize, 255);
    
    for (i = 0; i < ysize; i++) {
        fwrite(buf + wrap * i, 1, xsize, f);
    }
    //fwrite(buf, 1, xsize * ysize, f);
    
    fclose(f);
}

int main(int argc, char *argv[])
{
    AVFormatContext *pFmtCtx = avformat_alloc_context();
    
    if (avformat_open_input(&pFmtCtx, argv[1], NULL, NULL)) {
        fprintf(stderr, "open input failed\\n");
        return -1;
    }
    
    printf("Format:%s, duration:%ld us\\n", pFmtCtx->iformat->long_name, pFmtCtx->duration);
    
    if (avformat_find_stream_info(pFmtCtx, NULL) < 0) {
        fprintf(stderr, "find stream info failed\\n");
        return -2;
    }
    
    for (int i = 0; i < pFmtCtx->nb_streams; i++) {
        AVCodecParameters *pCodecPar = pFmtCtx->streams[i]->codecpar;
        AVCodec *pCodec = NULL;
        pCodec = avcodec_find_decoder(pCodecPar->codec_id);
        if (NULL == pCodec) {
            fprintf(stderr, "Unsupported codec!\\n");
            return -3;
        }
        if (AVMEDIA_TYPE_VIDEO == pCodecPar->codec_type) {
            printf("Video Codec: resolution: %d x %d\\n", pCodecPar->width, pCodecPar->height);
        } else if (AVMEDIA_TYPE_AUDIO == pCodecPar->codec_type) {
            printf("Audio Codec: %d channels, sample rate %d\\n", pCodecPar->channels, pCodecPar->sample_rate);
        }
        printf("\\t Codec %s ID %d bit_rate %ld\\n", pCodec->long_name, pCodec->id, pCodecPar->bit_rate);
        
        AVCodecContext *pCodecCtx = NULL;
        pCodecCtx = avcodec_alloc_context3(pCodec);
        avcodec_parameters_to_context(pCodecCtx, pCodecPar);
        
        if (avcodec_open2(pCodecCtx, pCodec, NULL)) {
            fprintf(stderr, "Codec open failed!\\n");
            return -4;
        }
        
        AVPacket *pPacket = av_packet_alloc();
        AVFrame *pFrame = av_frame_alloc();
        
        while (av_read_frame(pFmtCtx, pPacket) >= 0) {
            avcodec_send_packet(pCodecCtx, pPacket);
            avcodec_receive_frame(pCodecCtx, pFrame);
            printf("Frame %c (%d) pts %ld dts %ld key_frame %d [coded_picture_number %d, display_picture_number %d]\\n",
                    av_get_picture_type_char(pFrame->pict_type),
                    pCodecCtx->frame_number,
                    pFrame->pts,
                    pFrame->pkt_dts,
                    pFrame->key_frame,
                    pFrame->coded_picture_number,
                    pFrame->display_picture_number
            );
            if (AVMEDIA_TYPE_VIDEO == pCodecPar->codec_type) {
                char frame_filename[30] = {0};
                snprintf(frame_filename, 29, "./frame/Frame%d_%dx%d.yuv", pCodecCtx->frame_number, pFrame->width, pFrame->height);
                save_gray_frame(pFrame->data[0], pFrame->linesize[0], pFrame->width, pFrame->height, frame_filename);
            }
        }
    }
    
    return 0;
}