通过 tcp 套接字流式传输 PCM 音频

Posted 2023-02-25

技术标签:

【中文标题】通过 tcp 套接字流式传输 PCM 音频【英文标题】：Streaming PCM audio over tcp socket 【发布时间】：2020-01-05 11:48:11 【问题描述】：

我有一个来自 TCP 套接字的连续原始 PCM 音频数据流，我想播放它们。我做了很多研究，看到了很多样本，但没有结果。 This gist 是最接近的解决方案，但问题是，它是流式 mp3 文件。所以我有一个接收线性 PCM 音频数据并将它们提供给播放器的套接字，如下所示：

func play(_ data: Data) 
    // this function is called for every 320 bytes of linear PCM data.
    // play the 320 bytes of PCM data here!

那么有没有“简单”的方式来播放原始 PCM 音频数据？

【问题讨论】：

【参考方案1】：

对于 ios，您可以使用带有循环缓冲区的 RemoteIO 音频单元或 AVAudioEngine 进行实时音频流传输。

您不能将网络数据直接提供给音频输出，而应将其放在一个循环缓冲区中，音频子系统播放回调可以从该缓冲区以固定速率消耗它。您需要预先缓冲一些音频样本以覆盖网络抖动。

执行此操作的简单“方法”可能无法优雅地处理网络抖动。

【讨论】：

谢谢。我会搜索这些关键字。你知道有什么库或示例代码做类似的工作吗？【参考方案2】：

回答晚了，但如果您仍然卡在播放 TCP 字节，请尝试按照我的回答，将您的 tcp 音频字节放入循环缓冲区并通过 AudioUnit 播放。下面的代码从 TCP 接收字节并将它们放入 TPCircularBuffer

func tcpReceive() 
        receivingQueue.async 
            repeat 
                do 
                    let datagram = try self.tcpClient?.receive()
                    var byteData = datagram?["data"] as? Data
                    let dataLength = datagram?["length"] as? Int


                    let _ = TPCircularBufferProduceBytes(&self.circularBuffer, byteData.bytes, UInt32(decodedLength * 2))

                 catch 
                    fatalError(error.localizedDescription)
                
             while true

创建音频单元...

var desc = AudioComponentDescription(
            componentType: OSType(kAudioUnitType_Output),
            componentSubType: OSType(kAudioUnitSubType_VoiceProcessingIO),
            componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
            componentFlags: 0,
            componentFlagsMask: 0
        )

        let inputComponent = AudioComponentFindNext(nil, &desc)

        status = AudioComponentInstanceNew(inputComponent!, &audioUnit)
        if status != noErr 
            print("Audio component instance new error \(status!)")
        

 // Enable IO for playback
        status = AudioUnitSetProperty(
            audioUnit!,
            kAudioOutputUnitProperty_EnableIO,
            kAudioUnitScope_Output,
            kOutputBus,
            &flag,
            SizeOf32(flag)
        )
        if status != noErr 
            print("Enable IO for playback error \(status!)")
        

//Use your own format, I have sample rate of 16000 and pcm 16 Bit
        var ioFormat = CAStreamBasicDescription(
            sampleRate: 16000.0,
            numChannels: 1,
            pcmf: .int16,
            isInterleaved: false
        )     

//This is playbackCallback      var playbackCallback = AURenderCallbackStruct(
            inputProc: AudioController_PlaybackCallback, //This is a delegate where audioUnit puts the bytes
            inputProcRefCon: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
        )

        status = AudioUnitSetProperty(
            audioUnit!,
            AudioUnitPropertyID(kAudioUnitProperty_SetRenderCallback),
            AudioUnitScope(kAudioUnitScope_Input),
            kOutputBus,
            &playbackCallback,
            MemoryLayout<AURenderCallbackStruct>.size.ui
        )
        if status != noErr 
            print("Failed to set recording render callback \(status!)")
        
//Init Audio Unit
  status = AudioUnitInitialize(audioUnit!)
        if status != noErr 
            print("Failed to initialize audio unit \(status!)")
        

//Start AudioUnit      status = AudioOutputUnitStart(audioUnit!)
        if status != noErr 
            print("Failed to initialize output unit \(status!)")

这是我的播放回调函数，我从循环缓冲区播放音频

func performPlayback(
        _ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>,
        inTimeStamp: UnsafePointer<AudioTimeStamp>,
        inBufNumber: UInt32,
        inNumberFrames: UInt32,
        ioData: UnsafeMutablePointer<AudioBufferList>
    ) -> OSStatus 
        let buffer = ioData[0].mBuffers

        let bytesToCopy = ioData[0].mBuffers.mDataByteSize
        var bufferTail: UnsafeMutableRawPointer?

        var availableBytes: UInt32 = 0
        bufferTail = TPCircularBufferTail(&self.circularBuffer, &availableBytes)
        let bytesToWrite = min(bytesToCopy, availableBytes)


        var bufferList = AudioBufferList(
            mNumberBuffers: 1,
            mBuffers: ioData[0].mBuffers)

        var monoSamples = [Int16]()
        let ptr = bufferList.mBuffers.mData?.assumingMemoryBound(to: Int16.self)
        monoSamples.append(contentsOf: UnsafeBufferPointer(start: ptr, count: Int(inNumberFrames)))
        print(monoSamples)

        memcpy(buffer.mData, bufferTail, Int(bytesToWrite))
        TPCircularBufferConsume(&self.circularBuffer, bytesToWrite)

        return noErr

对于 TPCircularBuffer，我使用了这个 pod