在异步模式下使用 MediaCodec，问题是，我没有得到 MediaCodec.BUFFER_FLAG_END_OF_STREAM

Posted 2023-04-17

【中文标题】在异步模式下使用 MediaCodec，问题是，我没有得到 MediaCodec.BUFFER_FLAG_END_OF_STREAM

【英文标题】：Using MediaCodec in async mode, issue is, I am not getting MediaCodec.BUFFER_FLAG_END_OF_STREAM

【发布时间】：2020-10-05 22:14:29

【问题描述】：

我正在构建一个流媒体应用程序。我遇到了一个问题，这里是代码

我想将摄像头馈送直播到服务器，我希望我能在 onOutputBufferAvailable() 中获得ByteBuffer。我正在获取输出缓冲区，但当我调用 stopVideoCapture() 时，我从未得到 MediaCodec.BUFFER_FLAG_END_OF_STREAM

这里是代码段

创建媒体编解码器

 private val recorderStreamSurface by lazy 
    val format = MediaFormat.createVideoFormat(VIDEO_MIME_TYPE, width, height)
    val frameRate = 30 // 30 fps
    var recorderStreamSurface: Surface? = null
    // Set some required properties. The media codec may fail if these aren't defined.
    format.setInteger(
        MediaFormat.KEY_COLOR_FORMAT,
        MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface
    )
    format.setInteger(MediaFormat.KEY_BIT_RATE, 6000000) // 6Mbps
    format.setInteger(MediaFormat.KEY_FRAME_RATE, frameRate)
    format.setInteger(MediaFormat.KEY_CAPTURE_RATE, frameRate)
    format.setInteger(MediaFormat.KEY_REPEAT_PREVIOUS_FRAME_AFTER, 1000000 / frameRate)
    format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1)
    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1) // 1 seconds between I-frames
    videoEncoder = MediaCodec.createEncoderByType(VIDEO_MIME_TYPE)
    // Create a MediaCodec encoder and configure it. Get a Surface we can use for recording into.
    try 

        videoEncoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
        recorderStreamSurface = videoEncoder.createInputSurface()

         videoEncoder.setCallback(object : MediaCodec.Callback() 
             override fun onError(codec: MediaCodec, exception: MediaCodec.CodecException) 
                 Log.d(TAG, "==onError $codec $exception")
                 serverChannel.onError(exception)
             

             override fun onOutputFormatChanged(codec: MediaCodec, format: MediaFormat) 
                 Log.d(TAG, "video encoder: output format changed")
             

             override fun onInputBufferAvailable(codec: MediaCodec, index: Int) 
                 Log.d(TAG, "video encoder: returned input buffer: $index")
                 val frameData: ByteArray
                 frameData = queue.take().array()

                 val inputData = codec.getInputBuffer(index)
                 inputData!!.clear()
                 inputData.put(frameData)

                 codec.queueInputBuffer(index, 0, frameData.size, 0, 0)
             

             override fun onOutputBufferAvailable(codec: MediaCodec, index: Int, info: MediaCodec.BufferInfo) 
                 Log.d(TAG, "video encoder: returned output buffer: $index flag : $info.flags")
                 Log.d(TAG, "video encoder: returned buffer of size " + info.size)

                 if ((info.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) 
                     Log.i(TAG,"serverChannel.onCompleted()1")
                 

                 videoEncoder.releaseOutputBuffer(index, false)
             
         )
        videoEncoder.start()
     catch (e: IOException) 
        videoEncoder.stop()
        videoEncoder.release()
        serverChannel.onError(e)
    
    recorderStreamSurface

局部变量

lateinit var videoEncoder: MediaCodec
val queue: ArrayBlockingQueue<ByteBuffer> = ArrayBlockingQueue<ByteBuffer>(10)
val targets by lazy  listOf(viewFinder.holder.surface, recorderStreamSurface!!) 
private const val VIDEO_MIME_TYPE = "video/avc"
val cameraId = "1"
val fps = 30
val width = 1080
val height = 1920

记录请求

 private val recordRequest: CaptureRequest by lazy 
    // Capture request holds references to target surfaces
    session.device.createCaptureRequest(CameraDevice.TEMPLATE_RECORD).apply 
        // Add the preview and recording surface targets
        for (target: Surface in targets) 
            addTarget(target)
        
        // Sets user requested FPS for all targets
        set(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE, Range(fps, fps))
    .build()

最后开始和停止录制

private fun startVideoCapture() 

    // Prevents screen rotation during the video recording
    requireActivity().requestedOrientation =
        ActivityInfo.SCREEN_ORIENTATION_LOCKED
    session.setRepeatingRequest(previewRequest, null, cameraHandler)
    // Start recording repeating requests, which will stop the ongoing preview
    //  repeating requests without having to explicitly call `session.stopRepeating`
    session.setRepeatingRequest(recordRequest, null, cameraHandler)

    recordingStartMillis = System.currentTimeMillis()
    Log.d(TAG, "Recording started")



private fun stopVideoCapture() 
    // Unlocks screen rotation after recording finished
    requireActivity().requestedOrientation =
        ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED

    videoEncoder.stop()
    videoEncoder.release()
    Log.d(TAG, "Recording stopped")
    session.setRepeatingRequest(previewRequest, null, cameraHandler)

【问题讨论】：

我没有看到任何对 signalEndOfInputStream 的调用：developer.android.com/reference/android/media/…

【参考方案1】：

您必须将标志 BUFFER_FLAG_END_OF_STREAM 与最后要编码的数据一起作为参数传递。

codec.queueInputBuffer(index, 0, frameData.size, 0, BUFFER_FLAG_END_OF_STREAM)