将音频样本转换为 AVFrame FFmpeg & C++
Posted
技术标签:
【中文标题】将音频样本转换为 AVFrame FFmpeg & C++【英文标题】:Convert audio samples to AVFrame FFmpeg & C++ 【发布时间】:2021-01-30 12:17:35 【问题描述】:我真的需要帮助来纠正我目前正在使用的方法。该方法应将样本转换并写入ac3文件。
输入样本是 AV_SAMPLE_FMT_FLT 格式的 BYTE*
对于编码器,样本必须具有 AV_SAMPLE_FMT_FLTP 格式
bool AddAudiosample(AVFormatContext * pFormatContext, AVStream * pStream, BYTE * audiodata, uint32_t sampleCount, uint64_t devicets)
AVCodecContext * pCodecCxt = NULL;
bool res = true;
pCodecCxt = pStream->codec;
AVFrame* pFLTAudioFrame = NULL;
pFLTAudioFrame = av_frame_alloc();
AVFrame* pFLTPAudioFrame = NULL;
pFLTPAudioFrame = av_frame_alloc();
ProcessData(pFLTAudioFrame, pFLTPAudioFrame, (uint8_t *)audiodata, sampleCount, devicets);
swr_convert(pSmplConvertCtx, pFLTPAudioFrame->data, pFLTPAudioFrame->nb_samples, (const uint8_t **)pFLTAudioFrame->data, pFLTAudioFrame->nb_samples);
AVPacket pkt;
av_init_packet(&pkt);
pkt.flags |= AV_PKT_FLAG_KEY;
pkt.stream_index = pStream->index;
pkt.data = pAudioEncodeBuffer;
pkt.size = pFLTPAudioFrame->pkt_size;
int gotOutput = 0;
auto ret = avcodec_encode_audio2(pCodecCxt, &pkt, pFLTPAudioFrame, &gotOutput);
if (ret < 0)
exit(1);
if (gotOutput)
pkt.pts = av_rescale_q(pCodecCxt->coded_frame->pts, pCodecCxt->time_base, pStream->time_base);
ret = av_interleaved_write_frame(pFormatContext, &pkt);
if (ret < 0)
exit(1);
return res;
void ProcessData(AVFrame *inputframe, AVFrame *outputFrame, uint8_t* data, uint32_t sample_count, uint64_t device_ts)
inputframe->nb_samples = sample_count;
inputframe->format = AV_SAMPLE_FMT_FLT;
inputframe->sample_rate = mWFX->nSamplesPerSec;
inputframe->channels = mWFX->nChannels;
inputframe->pkt_size = sample_count*mWFX->nBlockAlign;
av_samples_fill_arrays(inputframe->data, inputframe->linesize, data, mWFX->nChannels, sample_count, AV_SAMPLE_FMT_FLT, 1);
outputFrame->nb_samples = inputframe->nb_samples;
outputFrame->format = AV_SAMPLE_FMT_FLTP;
outputFrame->sample_rate = inputframe->sample_rate;
outputFrame->channels = inputframe->channels;
outputFrame->pkt_size = sample_count*mWFX->nBlockAlign;
av_samples_fill_arrays(outputFrame->data, outputFrame->linesize, pAudioEncodeBuffer, mWFX->nChannels, sample_count, AV_SAMPLE_FMT_FLTP, 1);
这里是缓冲包的上下文重采样设置和计算:
pSmplConvertCtx = swr_alloc();
if (!pSmplConvertCtx)
fprintf(stderr, "Could not allocate resampler context\n");
exit(1);
av_opt_set_int (pSmplConvertCtx, "in_channel_count", pCodecCxt->channels, 0);
av_opt_set_int (pSmplConvertCtx, "in_sample_rate", pCodecCxt->sample_rate, 0);
av_opt_set_sample_fmt(pSmplConvertCtx, "in_sample_fmt", AV_SAMPLE_FMT_FLT, 0);
av_opt_set_int (pSmplConvertCtx, "out_channel_count", pCodecCxt->channels, 0);
av_opt_set_int (pSmplConvertCtx, "out_sample_rate", pCodecCxt->sample_rate, 0);
av_opt_set_sample_fmt(pSmplConvertCtx, "out_sample_fmt", pCodecCxt->sample_fmt, 0);
if ((swr_init(pSmplConvertCtx)) < 0)
fprintf(stderr, "Failed to initialize the resampling context\n");
exit(1);
nSizeAudioEncodeBuffer = av_samples_get_buffer_size(NULL, pCodecCxt->channels, pCodecCxt->sample_rate, pCodecCxt->sample_fmt, 1);
if (pAudioEncodeBuffer == NULL)
pAudioEncodeBuffer = (uint8_t * )av_malloc(nSizeAudioEncodeBuffer);
【问题讨论】:
不确定,这可能指向正确的方向:***.com/questions/45241185/… 我对 libav 不是很熟悉。 【参考方案1】:这里是过程函数:
process_data(AVFrame *frame, uint8_t* data, uint32_t sample_count, uint64_t device_ts)
int sample_size = _bit_per_sample / 8 * _channel_num;
//wasapi time unit is 100ns,so time base is NS_PER_SEC
frame->pts = _use_device_ts ? device_ts * 100 : av_gettime_relative();
if(_use_device_ts == false)
frame->pts -= (int64_t)sample_count * NS_PER_SEC / (int64_t)_sample_rate;
frame->pkt_dts = frame->pts;
frame->nb_samples = sample_count;
frame->format = _fmt;
frame->sample_rate = _sample_rate;
frame->channels = _channel_num;
frame->pkt_size = sample_count*sample_size;
av_samples_fill_arrays(frame->data, frame->linesize, data, _channel_num, sample_count, _fmt, 1);
if (_on_data) _on_data(frame, _cb_extra_index);
更多详情,您可以关注 https://github.com/peilinok/screen-recorder
【讨论】:
你的输出文件听起来像什么? 我正在尝试创建一个 AVI 文件。它有两个音频和视频流 视频写入正确。我有音频问题 输出文件没有声音。 您必须确保视频和音频流的pkt设置正确以上是关于将音频样本转换为 AVFrame FFmpeg & C++的主要内容,如果未能解决你的问题,请参考以下文章
如何将像素格式为 AV_PIX_FMT_CUDA 的 FFmpeg AVFrame 转换为像素格式为 AV_PIX_FMT_RGB 的新 AVFrame