带有 AVAudioConverterInputBlock 的 AVAudioConverter 处理后音频口吃

Posted 2023-02-15

【中文标题】带有 AVAudioConverterInputBlock 的 AVAudioConverter 处理后音频口吃

【英文标题】：AVAudioConverter with AVAudioConverterInputBlock stutters audio after processing

【发布时间】：2017-12-01 06:06:21

【问题描述】：

我正在尝试将音频缓冲区转换为不同的格式，并且我正在使用 AVAudioConverter。当您具有相同的采样率并且您不需要使用 AVAudioConverterInputBlock 时，AVAudioConverter 可以完成这项工作。

但如果我处理相同的采样率，我的音频数据就会出现奇怪的卡顿。我感觉我没有很好地处理输入块。输出中有重复两到三遍的单词。下面是完整的方法：

func sendAudio(audioFile: URL, completionHandler: @escaping (Bool, Bool, Data?)->Void) 

    createSession() sessionUrl, observeURL, session in
        let file = try! AVAudioFile(forReading: audioFile)
        let formatOfAudio = file.processingFormat
        self.engine = AVAudioEngine()
        guard let input = self.engine.inputNode else 
            print("no input")
            return
        
        //The audio in format in this case is: <AVAudioFormat 0x61800009d010:  2 ch,  44100 Hz, Float32, non-inter>
        let formatIn = formatOfAudio
        let formatOut = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: 16000, channels: 1, interleaved: true)
        let mixer = AVAudioMixerNode()
        self.engine.attach(mixer)
        mixer.volume = 0.0
        self.engine.attach(self.audioPlayerNode)
        self.engine.connect(self.audioPlayerNode, to: mixer, format: formatIn)
        self.engine.connect(input, to: mixer, format: input.outputFormat(forBus: 0))
        self.engine.connect(mixer, to: self.engine.mainMixerNode, format: formatIn)
        let audioConverter = AVAudioConverter(from: formatIn, to: formatOut)
        mixer.installTap(onBus: 0, bufferSize: 32000, format: formatIn, block: 
            (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
                let convertedBuffer = AVAudioPCMBuffer(pcmFormat: formatOut, frameCapacity: buffer.frameCapacity)
                let inputBlock: AVAudioConverterInputBlock =  inNumPackets, outStatus in
                    outStatus.pointee = AVAudioConverterInputStatus.haveData
                    return buffer
                
                var error: NSError? = nil
                let status = audioConverter.convert(to: convertedBuffer, error: &error, withInputFrom: inputBlock)
                let myData = convertedBuffer.toData()
                completionHandler(true, false, myData)
        )
        self.audioPlayerNode.scheduleFile(file, at: nil)
            self.delayWithSeconds(3.0)
            self.engine.stop()
            mixer.removeTap(onBus: 0)
            completionHandler(true, true, nil)
            
        
        do 
            try self.engine.start()
         catch 
            print(error)
        
        self.audioPlayerNode.play()
    

有什么想法吗？我从Apple slide sample 得到这个代码：

// Create an input block that’s called when converter needs input
let inputBlock : AVAudioConverterInputBlock = inNumPackets, outStatus in 
    if (<no_data_available>)    
        outStatus.memory = AVAudioConverterInputStatus.NoDataNow; 
        return nil;  
     else if (<end_of_stream>)    
        outStatus.memory = AVAudioConverterInputStatus.EndOfStream; 
        return nil;  
     else 
        ..outStatus.memory = AVAudioConverterInputStatus.HaveData;   
        return inBuffer; // fill and return input buffer 
      

【参考方案1】：

所以我相信我想通了。转换后的缓冲区帧容量必须除以被转换的采样率的比率。因此，完整的答案如下所示：

func sendAudio(audioFile: URL, completionHandler: @escaping (Bool, Bool, Data?)->Void) 

    createSession() sessionUrl, observeURL, session in
        let file = try! AVAudioFile(forReading: audioFile)
        let formatOfAudio = file.processingFormat
        self.engine = AVAudioEngine()
        guard let input = self.engine.inputNode else 
            print("no input")
            return
        
        //The audio in format in this case is: <AVAudioFormat 0x61800009d010:  2 ch,  44100 Hz, Float32, non-inter>
        let formatIn = formatOfAudio
        let formatOut = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: 16000, channels: 1, interleaved: true)
        let mixer = AVAudioMixerNode()
        self.engine.attach(mixer)
        mixer.volume = 0.0
        self.engine.attach(self.audioPlayerNode)
        self.engine.connect(self.audioPlayerNode, to: mixer, format: formatIn)
        self.engine.connect(input, to: mixer, format: input.outputFormat(forBus: 0))
        self.engine.connect(mixer, to: self.engine.mainMixerNode, format: formatIn)
        let audioConverter = AVAudioConverter(from: formatIn, to: formatOut)
        //Here is where I adjusted for the sample rate. It's hard coded here, but you would want to adjust so that you're dividing the input sample rate by your chosen sample rate.
        let sampleRateConversionRatio: Float = 44100.0/16000.0

        mixer.installTap(onBus: 0, bufferSize: 32000, format: formatIn, block: 
        (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
                //And this is where you set the appropriate capacity!
                let capacity = UInt32(Float(buffer.frameCapacity)/ratio)
                let convertedBuffer = AVAudioPCMBuffer(pcmFormat: formatOut, frameCapacity: capacity)
                let inputBlock: AVAudioConverterInputBlock =  inNumPackets, outStatus in
                    outStatus.pointee = AVAudioConverterInputStatus.haveData
                    return buffer
                
                var error: NSError? = nil
                let status = audioConverter.convert(to: convertedBuffer, error: &error, withInputFrom: inputBlock)
                let myData = convertedBuffer.toData()
                completionHandler(true, false, myData)
        )
        self.audioPlayerNode.scheduleFile(file, at: nil)
            self.delayWithSeconds(3.0)
            self.engine.stop()
            mixer.removeTap(onBus: 0)
            completionHandler(true, true, nil)
            
        
        do 
            try self.engine.start()
         catch 
            print(error)
        
        self.audioPlayerNode.play()
    

【讨论】：

我在使用这个转换器时遇到了类似的问题，文档确实缺乏这样的任何细节。其他人可能还需要注意，尽管您有bufferSize: 32000，但要计算容量，您会使用buffer.frameCapacity 获得实际的缓冲区frameCapacity。您不能假设它实际上会使用您请求的缓冲区大小 - 它可以忽略您。我尝试使用更大的大小（48000），实际缓冲区为19200，所以你必须做你所做的！

【参考方案2】：

对于任何发现此问题的人来说，真正的根本原因是不正确地使用 AVAudioConverterInputBlock。目标缓冲区的容量无关紧要，只要它足够大即可，但是会重复调用该块，直到目标缓冲区被填满。

如果您的源缓冲区包含ABC，它将用ABCABCABC... 填充目标。然后，如果您将其通过管道传输到实时播放，则会随机切断块以适应播放时间，从而产生这种奇怪的噼啪声。

实际的解决方案是在缓冲区提交给转换器后将AVAudioConverterInputStatus 正确设置为.noDataNow。请注意，返回 .endOfStream 将永远锁定转换器对象。

var gotData = false
self.converter.convert(to: convertedBuffer, error: nil, withInputFrom:  (_, outStatus) -> AVAudioBuffer? in
    if gotData 
        outStatus.pointee = .noDataNow
        return nil
    
    gotData = true
    outStatus.pointee = .haveData
    return inputBuffer
)            

【讨论】：

我试过了，效果很好。总是很高兴得到正确的正确答案，而不是 hack。