将 AAC 的 avcodec 配置配置为 HLS/MPEG-TS([mpegts @ 0x7fc4c00343c0] AAC 比特流不是 ADTS 格式并且缺少额外数据)

Posted

技术标签:

【中文标题】将 AAC 的 avcodec 配置配置为 HLS/MPEG-TS([mpegts @ 0x7fc4c00343c0] AAC 比特流不是 ADTS 格式并且缺少额外数据)【英文标题】:avcodec config for AAC into HLS/MPEG-TS ([mpegts @ 0x7fc4c00343c0] AAC bitstream not in ADTS format and extradata missing) 【发布时间】:2021-04-08 20:30:28 【问题描述】:

我们正在将视频编码为 H264,并将原始 PCM 样本编码为 AAC,以用于 HLS 流式传输。视频运行良好,但在 libavcodec 中配置 AAC 编码器时遇到问题。

This SO question 说:

有两种方法可以将 AAC 放入传输流中。

1.使用 ADTS 语法(MPEG2 风格)

在这种情况下,PMT 的 stream_type 应指定为 0x0F(ISO/IEC 13818-7 带有 ADTS 传输语法的音频)。

因此,您只能使用“旧”(MPEG2) AAC 版本,而不能使用 SBR 和 PS。

2。使用 LATM+LOAS/AudiosyncStream 语法(MPEG4 风格)。

在这种情况下,PMT 的 stream_type 应指定为 0x11(ISO/IEC 14496-3 使用 LATM 传输语法的音频)。

您可以使用“新”(MPEG4) AAC 功能的所有力量,包括 SBR 和 PS。

此外,DVB 标准 ETSI TS 101 154 要求:HEv1/HEv2 AAC 应 使用 LATM 语法传输。

但经过大量搜索后,我找不到任何有关如何执行其中任何一项的文档。在将编码音频传递到 MPEG-TS 多路复用器(用于输出到 HLS)之前,下面的配置缺少什么来获取带有 ADTS 或 LATM 的编码音频?

设置 AAC 编解码器的当前代码给出错误[mpegts @ 0x7fc4c00343c0] AAC bitstream not in ADTS format and extradata missing

AAC 编码器设置(为简洁起见,删除了错误检查)

/// Set up Encoder ///
mpAudioCodec = avcodec_find_encoder(AV_CODEC_ID_AAC);
mpAudioCodecContext = avcodec_alloc_context3(mpAudioCodec);

mpAudioCodecContext->bit_rate       = DEFAULT_AUD_BITRATE;
mpAudioCodecContext->sample_rate    = DEFAULT_AUD_SAMPLE_RATE;
mpAudioCodecContext->channel_layout = DEFAULT_AUD_CHAN_LAYOUT;
mpAudioCodecContext->channels       = 2; 
mpAudioCodecContext->sample_fmt     = AV_SAMPLE_FMT_FLTP; // S16 not supported. Must convert

mpAudioCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

rc = avcodec_open2(mpAudioCodecContext, mpAudioCodec, 0);

HLS 多路复用器设置

avformat_alloc_output_context2(&mpOutputMux, 0, "hls", path.c_str());

// VIDEO TRACK
mpVideoTrack = avformat_new_stream(mpOutputMux, 0);
mpVideoTrack->id = 0;
mpVideoTrack->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
mpVideoTrack->codecpar->codec_id   = AV_CODEC_ID_H264;

mpVideoTrack->time_base      = (AVRational)  1,          mFrameRate ;
mpVideoTrack->avg_frame_rate = (AVRational)  mFrameRate, 1          ;

// AUDIO TRACK
mpAudioTrack = avformat_new_stream(mpOutputMux, 0);
mpAudioTrack->id = 1;

mpAudioTrack->codecpar->codec_type  = AVMEDIA_TYPE_AUDIO;
mpAudioTrack->codecpar->codec_id    = DEFAULT_AUDIO_CODEC;
mpAudioTrack->codecpar->sample_rate = mpAudioCodecContext->sample_rate;

mpAudioTrack->time_base.den = mpAudioCodecContext->sample_rate;
mpAudioTrack->time_base.num = 1;

AVDictionary *hlsOptions = NULL;
av_dict_set(&hlsOptions,     "hls_segment_type",   "mpegts", 0);
av_dict_set(&hlsOptions,     "segment_list_type",  "m3u8",   0);
av_dict_set_int(&hlsOptions, "hls_list_size",      mPlaylistSize,  0);
av_dict_set_int(&hlsOptions, "hls_time",           mChunkDurSec,   0);
av_dict_set(&hlsOptions,     "hls_flags",          "delete_segments", 0);
av_dict_set(&hlsOptions,     "hls_segment_filename", segPath.c_str(),   0);

av_dict_set_int(&hlsOptions, "reference_stream",   mpVideoTrack->index, 0);
av_dict_set(&hlsOptions,     "segment_list_flags", "cache+live", 0);

int ret = avformat_write_header(mpOutputMux, &hlsOptions);

编码循环

int bytesCopied = mAudEsBuffer.popData(mpPcmS16Buf, mpPcmAudioFrame->nb_samples);

// resample to float
int rc = swr_convert(mpAudioResampleCtx, mpPcmAudioFrame->data, mpPcmAudioFrame->nb_samples, (const uint8_t**) &mpPcmS16Buf, mpPcmAudioFrame->nb_samples);

/* Set a timestamp based on the sample rate for the container. */
mCurAudPts += mpPcmAudioFrame->nb_samples;
mpPcmAudioFrame->pts = mCurAudPts;

// send frame for encoding to AAC
rc = avcodec_send_frame(mpAudioCodecContext, mpPcmAudioFrame);

/* read all the available output packets (in general there may be any number of them */
while (rc >= 0)

    // need to init packet every time??
    /* Set the packet data and size so that it is recognized as being empty. */
    av_init_packet(mpEncAudioPacket);
    mpEncAudioPacket->data = NULL;
    mpEncAudioPacket->size = 0;

    rc = avcodec_receive_packet(mpAudioCodecContext, mpEncAudioPacket);
    if (rc < 0)
    
        printf("TqHlsLib::readAndMuxAudio() - Error encoding audio frame: %s\n", av_make_error_string(mpErr, TQERRLEN, rc));
        return HLS_DEC_ERROR;
    

    TRACE(("%T %t TqHlsLib::readAndMuxAudio() - Got an encoded audio packet. %u bytes\n",
        mpEncAudioPacket->size ));

    /* rescale output packet timestamp values from codec to stream timebase */
    av_packet_rescale_ts(mpEncAudioPacket, mpAudioTrack->time_base, mpAudioTrack->time_base);
    mpEncAudioPacket->stream_index = mpAudioTrack->index;

    /* Write the compressed frame to the media file. */
    rc = av_interleaved_write_frame(mpOutputMux, mpEncAudioPacket);
    if (rc < 0)
    
        fprintf(stderr, "TqHlsLib::addVideoH264Packet - Error while writing audio packet: %s\n",
            av_make_error_string(mpErr, TQERRLEN, ret));

        // return some error here
    
    av_packet_unref(mpEncAudioPacket);


输出

[mpegts @ 0x7fb280144e00] AAC bitstream not in ADTS format and extradata missing
20:24:52.327418 24388 TqHlsLib::readAndMuxAudio() - Got an encoded audio packet. 185 bytes
[mpegts @ 0x7fb280144e00] AAC bitstream not in ADTS format and extradata missing
20:24:52.372975 24388 TqHlsLib::readAndMuxAudio() - Got an encoded audio packet. 188 bytes
[mpegts @ 0x7fb280144e00] AAC bitstream not in ADTS format and extradata missing

【问题讨论】:

【参考方案1】:

经过大量搜索,我找到了两种将 ADTS 标头添加到音频的方法,以便正确地 MUX MPEG-TS。

第一个是为 ADTS 设置一个单独的AVFormatContext,使用编码的 AAC 数据包创建一个AVStream,然后使用send_frame / receive_frame 获取相同的 AAC 数据,但这次使用 ADTS已应用标头。

这增加了很多复杂性和延迟。

最后,我只是在每个编码的 AAC 数据包中手动添加了一个 ADTS 标头,然后再传递给av_interleaved_write_frame

为了帮助以后的人,这里是代码:

/*
    ADTS HEADER: 7 Bytes. See ISO 13818-7 (2004)

    AAAAAAAA AAAABCCD EEFFFFGH HHIJKLMM MMMMMMMM MMMOOOOO OOOOOOPP

    A - Sync 0xFFFx
    B   1   MPEG Version: 0 for MPEG-4, 1 for MPEG-2
    C   2   Layer: always 0
    D   1   protection absent, Warning, set to 1 if there is no CRC and 0 if there is CRC
    E   2   profile, the MPEG-4 Audio Object Type minus 1
    F   4   MPEG-4 Sampling Frequency Index (15 is forbidden)
    G   1   private bit, guaranteed never to be used by MPEG, set to 0 when encoding, ignore when decoding
    H   3   MPEG-4 Channel Configuration (in the case of 0, the channel configuration is sent via an inband PCE)
    I   1   originality, set to 0 when encoding, ignore when decoding
    J   1   home, set to 0 when encoding, ignore when decoding
    K   1   copyrighted id bit, the next bit of a centrally registered copyright identifier, set to 0 when encoding, ignore when decoding
    L   1   copyright id start, signals that this frame's copyright id bit is the first bit of the copyright id, set to 0 when encoding, ignore when decoding
    M   13  frame length, this value must include 7 or 9 bytes of header length: FrameLength = (ProtectionAbsent == 1 ? 7 : 9) + size(AACFrame)
    O   11  Buffer fullness
    P   2   Number of AAC frames (RDBs) in ADTS frame minus 1, for maximum compatibility always use 1 AAC frame per ADTS frame
    Q   16  CRC if protection absent is 0

*/
void
addADTSHeader(uint8_t *inBuf, size_t inLen, const ADTSInfo &adtsInfo, std::vector<uint8_t> &outVec)

    outVec.clear();

    uint8_t b;

    // 0: Sync I
    outVec.push_back(0xFF);

    // 1: Sync II + BCCD
    b  = 0xF0;
    b |= ((adtsInfo.mpegVersion == 2) ? 0 : 1) << 3;
    b |= 1; // no protection
    outVec.push_back(b);

    // 2: EEFFFFGH
    int sampleFreqIdx = 4; // default 44100
    int i = 0;
    while (ADTS_SAMPLE_RATES[i] != LAST_INDEX)
    
        if (ADTS_SAMPLE_RATES[i] == adtsInfo.sampleRate)
        
            sampleFreqIdx = i;
            break;
        
        i++;
    

    b  = (adtsInfo.objectType - 1) << 6; // EE
    b |= sampleFreqIdx << 2;
    b |= (adtsInfo.channelConfig & 0x07) >> 2; // high bit of channel config
    outVec.push_back(b);

    // 3: HHIJKLMM
    b  = (adtsInfo.channelConfig & 0x03) << 6; // low two bits of channel config
    b |= ((7 + inLen) >> 11) & 0x03;           // high two bits of frame length
    outVec.push_back(b);

    // 4: MMMMMMMM
    b  = (7 + inLen) >> 3;                     // Frame length middle
    outVec.push_back(b);

    // 5: MMMOOOOO
    b  = ((7 + inLen) & 0x07) << 5;            // low three bits of frame length
    b |= 0x1F;                                 // buffer fullness 0x7FF VBR -> N/A
    outVec.push_back(b);

    // 6: OOOOOOPP
    b  = 0x3F << 2;                            // buffer fullness 0x7FF VBR -> N/A
    b |= 0;                                    // 1 AAC frame per ADTS
    outVec.push_back(b);

    outVec.insert(outVec.end(), inBuf, inBuf + inLen);

【讨论】:

以上是关于将 AAC 的 avcodec 配置配置为 HLS/MPEG-TS([mpegts @ 0x7fc4c00343c0] AAC 比特流不是 ADTS 格式并且缺少额外数据)的主要内容,如果未能解决你的问题,请参考以下文章

HLS点播实现(H.264和AAC码流)

如何在生成 HLS 段的同时优化 FFMPEG h264/aac 转换

以编程方式创建和更新 HLS 播放列表

FFmpeg-打开和配置音视频解码器(avcodec_find_decoder()avcodec_alloc_context3())

Linux下使用FFmpeg将RTMP流转换为HLS

Nginx RTMP配置文件种HLS部分