Wowza/WebRTC - 如何在相机翻转时重新协商视频格式

Posted 2023-03-04

【中文标题】Wowza/WebRTC - 如何在相机翻转时重新协商视频格式

【英文标题】：Wowza/WebRTC - how to re-negotiate video format on camera flip

【发布时间】：2021-08-03 19:06:51

【问题描述】：

我遵循了来自 here 的一些提示，在 ios 上通过 WebRTC 呈现/流式传输时让相机从前到后翻转。翻转本身有效，但是当RTCMediaStream 中的视频轨道发生变化时，RTCPeerConnection 再次调用peerConnectionShouldNegotiate 委托方法。这是我当前的实现（它适用于前置摄像头视频的初始调用，但在视频翻转时第二次调用时失败）：

func peerConnectionShouldNegotiate(_ peerConnection: RTCPeerConnection) 
        guard let stream = liveStream else 
            return
        
        print("peerConnectionShouldNegotiate")
        peerConnection.offer(for: mediaConstraints)  (desc, err) in
            if let error = err 
                print("peerConnectionShouldNegotiate error", error)
             else if let desc = desc 
                self.con?.setLocalDescription(desc, completionHandler:  (err) in
                    if let error = err 
                        print("set local desc error", error)
                     else 
                        let message: [String: Any] = [
                            "direction": "publish",
                            "command": "sendOffer",
                            "streamInfo": [
                                "applicationName": stream.appName,
                                "streamName": stream.streamName,
                                "sessionId": self.sessionId
                            ],
                            "sdp": [
                                "type": RTCSessionDescription.string(for: desc.type),
                                "sdp": desc.sdp
                            ],
                            "userData": [
                                "param1": "value1"
                            ]
                        ]
                        let messageData = try! JSONSerialization.data(withJSONObject: message, options: [])
                        let messageStr = String(data: messageData, encoding: .utf8)!
                        self.socket?.write(string: messageStr)
                    
                )
             else 
                print("this is a no no")
            
        
    

来自信令服务器的 web-socket 响应带有错误和消息Stream name is already in use: [my-stream-id]

我不知道应该向 Wowza 的信令服务器发送什么样的消息来更新现有流中的视频格式。看来sendOffer 试图开始新的蒸汽。很难找到任何关于此的文档。

【参考方案1】：

如果其他人遇到同样的问题，请在此处发布我的解决方案，我需要几天的反复试验才能得出解决方案。

我真正需要的只是切换相机而不重新协商RTCPeerConnection。我看到他们在 RTCRtpSender 上切换视频轨道的 javascript 示例，但这在 iOS 上不起作用，iOS 框架也没有浏览器的 javascript WebRTC 库中存在的 replaceVideoTrack 方法。

我的解决方案是在现有的RTCCameraVideoCapturer 上更新摄像头，从而保留现有的 RTCVideoSource 和RTCVideoTrack。

我最初创建视频轨道的电话是这样的：

    func createVideoTrack() -> RTCVideoTrack? 
        let devices = RTCCameraVideoCapturer.captureDevices()
        guard
            let camera = selectedCamera(devices: devices)
        else 
            return nil
        
        let source = rtcPCFactory.videoSource()
        let format = selectFormat(formats: camera.formats)
        let dims = CMVideoFormatDescriptionGetDimensions(format.formatDescription)
        if dims.width > dims.height 
            videoSize = CGSize(width: CGFloat(dims.height), height: CGFloat(dims.width))
         else 
            videoSize = CGSize(width: CGFloat(dims.width), height: CGFloat(dims.height))
        
        let fps = selectFrameRate(ranges: format.videoSupportedFrameRateRanges)
        source.adaptOutputFormat(toWidth: 512, height: 288, fps: Int32(fps))
        let videoTrack = rtcPCFactory.videoTrack(with: source, trackId: internalStreamId + "v0")

        let capturer = RTCCameraVideoCapturer(delegate: source)
        capturer.startCapture(with: camera, format: format, fps: Int(fps))
        
        cameraPreviewView.captureSession = capturer.captureSession
        videoCapturer = capturer
        return videoTrack
    

它成功地协商了对等连接，然后在翻转相机时我这样做了：

    func updateCapturerCamera() 
        let devices = RTCCameraVideoCapturer.captureDevices()
        guard
            let camera = selectedCamera(devices: devices),
            let capturer = videoCapturer
        else 
            return
        
        let format = selectFormat(formats: camera.formats)
        let dims = CMVideoFormatDescriptionGetDimensions(format.formatDescription)
        if dims.width > dims.height 
            videoSize = CGSize(width: CGFloat(dims.height), height: CGFloat(dims.width))
         else 
            videoSize = CGSize(width: CGFloat(dims.width), height: CGFloat(dims.height))
        
        let fps = selectFrameRate(ranges: format.videoSupportedFrameRateRanges)
        capturer.startCapture(with: camera, format: format, fps: Int(fps))
    

...并且相机翻转在实时视频流中发挥了魅力，无需重新协商连接。