在 ARKit 上显示具有透明背景的视频

Posted 2023-03-10

【中文标题】在 ARKit 上显示具有透明背景的视频

【英文标题】：Show a video with transparent background on ARKit

【发布时间】：2019-04-14 23:25:18

【问题描述】：

我试图简单地显示一个视频，删除其背景的一部分，这意味着是透明的。我已经尝试了 Stack Overflow 提供的几种方法，但到目前为止它们都低于标准。与我正在寻找的结果最接近的是这个链接中的那个：ARKit / SpriteKit - set pixelBufferAttributes to SKVideoNode or make transparent pixels in video (chroma-key effect) another way

然而，尽管这种方法是明智的，而且它似乎对提出问题的人有效，但在我的情况下，它只会使整个视频变白。

我的策略是显示视频并添加效果如下：

func setVideoNode(named name: String, in node: SCNNode, with imageReference: ARReferenceImage?, size: CGSize = CGSize(width: 500, height: 320), extension ext: String = "mp4") 

        let nodeWidth = imageReference!.physicalSize.width
        let nodeHeigth = imageReference!.physicalSize.height

        guard let videoUrl = Bundle.main.url(forResource: name, withExtension: ext) else 
            print("Guard Fail")
            return
        

        self.currentPlayer = AVPlayer(url: videoUrl)

        let videoNode = SKVideoNode(avPlayer: self.currentPlayer)
        videoNode.size = size
        videoNode.name = name
        videoNode.yScale = -1.0
        videoNode.play()

        let effectNode = SKEffectNode()
        effectNode.addChild(videoNode)
        effectNode.filter = colorCubeFilterForChromaKey(hueAngle: 0)

        let planeGeometry = SCNPlane(width: nodeWidth, height: nodeHeigth)
        planeGeometry.firstMaterial?.diffuse.contents = effectNode
        planeGeometry.firstMaterial?.isDoubleSided = true

        let planeNode = SCNNode()
        planeNode.geometry = planeGeometry
        planeNode.position = SCNVector3(planeNode.position.x + 1, 0.5, 0)
        planeNode.eulerAngles.x = -.pi / 2

        node.addChildNode(planeNode)

        self.currentPlayer.play()


    

func RGBtoHSV(r : Float, g : Float, b : Float) -> (h : Float, s : Float, v : Float) 
        var h : CGFloat = 0
        var s : CGFloat = 0
        var v : CGFloat = 0
        let col = UIColor(red: CGFloat(r), green: CGFloat(g), blue: CGFloat(b), alpha: 1.0)
        col.getHue(&h, saturation: &s, brightness: &v, alpha: nil)
        return (Float(h), Float(s), Float(v))
    

    func colorCubeFilterForChromaKey(hueAngle: Float) -> CIFilter 

        let hueRange: Float = 20 // degrees size pie shape that we want to replace
        let minHueAngle: Float = (hueAngle - hueRange/2.0) / 360
        let maxHueAngle: Float = (hueAngle + hueRange/2.0) / 360

        let size = 64
        var cubeData = [Float](repeating: 0, count: size * size * size * 4)
        var rgb: [Float] = [0, 0, 0]
        var hsv: (h : Float, s : Float, v : Float)
        var offset = 0

        for z in 0 ..< size 
            rgb[2] = Float(z) / Float(size) // blue value
            for y in 0 ..< size 
                rgb[1] = Float(y) / Float(size) // green value
                for x in 0 ..< size 

                    rgb[0] = Float(x) / Float(size) // red value
                    hsv = RGBtoHSV(r: rgb[0], g: rgb[1], b: rgb[2])
                    // TODO: Check if hsv.s > 0.5 is really nesseccary
                    let alpha: Float = (hsv.h > minHueAngle && hsv.h < maxHueAngle && hsv.s > 0.5) ? 0 : 1.0

                    cubeData[offset] = rgb[0] * alpha
                    cubeData[offset + 1] = rgb[1] * alpha
                    cubeData[offset + 2] = rgb[2] * alpha
                    cubeData[offset + 3] = alpha
                    offset += 4
                
            
        
        let b = cubeData.withUnsafeBufferPointer  Data(buffer: $0) 
        let data = b as NSData

        let colorCube = CIFilter(name: "CIColorCube", parameters: [
            "inputCubeDimension": size,
            "inputCubeData": data
            ])
        return colorCube!
    

就我而言，我正在尝试去除红色。有没有其他方法可以做到这一点？

【参考方案1】：

您好，亲爱的朋友，正如您在此 link 中看到的那样 .你必须在330 to 360 degree之间使用no来去除红色。

effectNode.filter = colorCubeFilterForChromaKey(hueAngle: 330)