AVAssetWriter - 设置自定义帧率

Posted 2023-03-08

【中文标题】AVAssetWriter - 设置自定义帧率

【英文标题】：AVAssetWriter - Set Custom frame rate

【发布时间】：2019-01-30 07:26:52

【问题描述】：

我正在使用 AVAssetWriter 使用委托从 ARSession 写入视频帧。

func session(_ session: ARSession, didUpdate frame: ARFrame)

请参阅下面用于编写图像的代码。

如何根据需要设置自定义帧速率，例如 24、30 或 60 等。

在输出设置中，为 AVVideoExpectedSourceFrameRateKey 提供的值为 30。但是我们为它提供的任何值，在检查 VLC 播放器 -> 媒体信息 -> 编解码器详细信息时，总是将“帧速率”设为 60

我应该进行哪些更改来设置所需的帧速率？提前致谢。

func writeImage(_ image: CVPixelBuffer, thisTimestamp: TimeInterval) 

        guard let videoDirector = videoWriter else  return 

        serialQueue.async(execute: 

            let scale = CMTimeScale(NSEC_PER_SEC)

            if (!self.seenTimestamps.contains(thisTimestamp)) 

                self.seenTimestamps.append(thisTimestamp)
                let pts = CMTime(value: CMTimeValue((thisTimestamp) * Double(scale)),
                                 timescale: scale)
                var timingInfo = CMSampleTimingInfo(duration: kCMTimeInvalid,
                                                    presentationTimeStamp: pts,
                                                    decodeTimeStamp: kCMTimeInvalid)

                var vidInfo:CMVideoFormatDescription! = nil
                CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, image, &vidInfo)

                var sampleBuffer:CMSampleBuffer! = nil
                CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault, image, true, nil, nil, vidInfo, &timingInfo, &sampleBuffer)

                let imageBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!

                if self.videoWriterInput == nil 

                    let width = CVPixelBufferGetWidth(imageBuffer)
                    let height = CVPixelBufferGetHeight(imageBuffer)


                    let numPixels: Double = Double(width * height);
                    let bitsPerPixel = 11.4;
                    let bitsPerSecond = Int(numPixels * bitsPerPixel)

                    // add video input
                    let outputSettings: [String: Any] = [
                        AVVideoCodecKey : AVVideoCodecType.h264,
                        AVVideoWidthKey : width,
                        AVVideoHeightKey : height,
                        AVVideoCompressionPropertiesKey : [
                            AVVideoExpectedSourceFrameRateKey: 30,
                            AVVideoAverageBitRateKey : bitsPerSecond,
                            AVVideoMaxKeyFrameIntervalKey : 1
                        ]
                    ]
                    self.videoWriterInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: outputSettings)
                    self.videoWriterInput?.expectsMediaDataInRealTime = true
                    guard let input = self.videoWriterInput else  return 

                    if videoDirector.canAdd(input) 
                        videoDirector.add(input)
                    
                    videoDirector.startWriting()
                

                let writable = self.canWrite()
                if writable, self.sessionAtSourceTime == nil 
                    let timeStamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
                    self.sessionAtSourceTime = timeStamp
                    videoDirector.startSession(atSourceTime: timeStamp)
                

                if self.videoWriterInput?.isReadyForMoreMediaData == true 
                    let appendResult = self.videoWriterInput?.append(sampleBuffer)
                    if appendResult == false 
                        printDebug("writer status: \(videoDirector.status.rawValue)")
                        printDebug("writer error: \(videoDirector.error.debugDescription)")
                    
                
            
        )
    
    func canWrite() -> Bool 
        return isRecording && videoWriter?.status == .writing
    

【问题讨论】：

AVVideoExpectedSourceFrameRateKey 的文档说“这不是用来控制帧速率的；它是作为视频编码器的提示提供的。也许 CMSampleTimingInfo.duration 需要设置为所需的帧速率？

@jpulikkottil ，您找到解决上述问题的方法了吗？

我们可以做的是启动一个计时器并从 ARCamera 获取图像。因此，如果您需要每秒 60 帧，则启动一个间隔为 1/60 的计时器。但除非必要，否则不建议这样做。

【参考方案1】：

Apple 支持部门的回复：

如果您想使用 AVAssetWriter 实际更改电影的帧速率，那么您必须在写出每个视频帧时正确设置它们的时间戳。

一种方法是使用 AVAssetWriterInputPixelBufferAdaptor 对象。例如，您使用 AVAssetWriterInputPixelBufferAdaptor 将打包为 CVPixelBuffer 对象的视频样本附加到单个 AVAssetWriterInput 对象。

初始化代码如下所示：

[dictionary setObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(NSString*) kCVPixelBufferPixelFormatTypeKey];

...

AVAssetWriterInputPixelBufferAdaptor *avAdaptor = [[AVAssetWriterInputPixelBufferAdaptor alloc] initWithAssetWriterInput:assetWriterInput sourcePixelBufferAttributes:dictionary];

然后，要为新电影指定特定的播放帧速率（例如 15 fps），时间戳（表示为 CMTime 结构）应指定为相隔 1/15 秒。这是一个代码sn-p：

CMTime frameTime = CMTimeMake(1, 15);
result = [avAdaptor appendPixelBuffer:buffer withPresentationTime:frameTime];

另一种方法是使用 CMSampleBufferCreateCopyWithNewTiming 重新定时缓冲区，然后将它们传递给 AVAssetWriter。这是一个粗略的大纲：

CMSampleTimingInfo sampleTimingInfo = 0;
CMSampleBufferRef newBuffer = NULL;

CMSampleBufferGetSampleTimingInfo(existingSampleBuffer, 0, &sampleTimingInfo);

sampleTimingInfo.duration = CMTimeMake(1, 30) // Specify new frame rate.
sampleTimingInfo.presentationTimeStamp =          CMTimeAdd(previousPresentationTimeStamp, sampleTimingInfo.duration);
previousPresentationTimeStamp = sampleTimingInfo.presentationTimeStamp;

OSStatus status = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, existingSampleBuffer, 1, &sampleTimingInfo, &newBuffer);

if (status == noErr) 
/* Write your newBuffer here */

//////////

我尝试使用“CMSampleBufferCreateCopyWithNewTiming”。 FPS 设置正确。但是得到了慢动作输出。以下是我的代码。

autoreleasepool 
                        //set FPS
                        var sampleTimingInfo: CMSampleTimingInfo = CMSampleTimingInfo(duration: kCMTimeInvalid, presentationTimeStamp: CMTime(), decodeTimeStamp: kCMTimeInvalid)
                        var newBuffer: CMSampleBuffer!  = nil

                        CMSampleBufferGetSampleTimingInfo(sampleBufferMain, 0, &sampleTimingInfo);

                        sampleTimingInfo.duration = CMTimeMake(1, 15) // Specify new frame rate.
                        sampleTimingInfo.presentationTimeStamp = CMTimeAdd(self.previousPresentationTimeStamp, sampleTimingInfo.duration)
                        self.previousPresentationTimeStamp = sampleTimingInfo.presentationTimeStamp

                        let status: OSStatus = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, sampleBufferMain, 1, &sampleTimingInfo, &newBuffer);

                        if status == noErr 
                            let appendResult = self.videoWriterInput?.append(newBuffer)
                            if appendResult == false 
                                printError("writer status: \(String(describing: self.videoWriter?.status.rawValue))")
                                printError("writer error: \(self.videoWriter?.error.debugDescription ?? "")")
                            
                         else 
                            print("write error")