ARKit + SceneKit：将重建场景用于物理？

Posted 2023-03-23

【中文标题】ARKit + SceneKit：将重建场景用于物理？

【英文标题】：ARKit + SceneKit: Using reconstructed scene for physics?

【发布时间】：2021-07-22 04:44:58

【问题描述】：

我将 ARKit 与 SceneKit 结合使用，并希望让我的 3D 对象与带有 LiDAR 传感器 (config.sceneReconstruction = .mesh) 的设备创建的 reconstructed scene 进行物理交互。例如，让虚拟球从重建场景的几何体中反弹。

在 RealityKit 中，这似乎可以使用 sceneUnderstanding：

arView.environment.sceneUnderstanding.options.insert(.physics)

使用 SceneKit 时如何实现同样的效果？

【参考方案1】：

据我所知，没有内置支持使用 SceneKit 启用此功能。但是，您可以使用 ARKit 创建的 ARMeshAnchor 轻松组合自定义解决方案。

首先，配置 ARKit 以启用场景重建：

let config = ARWorldTrackingConfiguration()
if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) 
    config.sceneReconstruction = .mesh
 else 
    // Handle device that doesn't support scene reconstruction


// and enable physics visualization for debugging
sceneView.debugOptions = [.showPhysicsShapes]

然后在您的ARSCNViewDelegate 中使用renderer nodeFor 为新创建的ARMeshAnchor 实例创建一个scenekit 节点：

func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? 
    guard let meshAnchor = anchor as? ARMeshAnchor else 
        return nil
    

    let geometry = createGeometryFromAnchor(meshAnchor: meshAnchor)

    // Optionally hide the node from rendering as well
    geometry.firstMaterial?.colorBufferWriteMask = []

    let node = SCNNode(geometry: geometry)

    // Make sure physics apply to the node
    // You must used concavePolyhedron here!
    node.physicsBody = SCNPhysicsBody(type: .static, shape: SCNPhysicsShape(geometry: geometry, options: [.type: SCNPhysicsShape.ShapeType.concavePolyhedron]))

    return node


// Taken from https://developer.apple.com/forums/thread/130599
func createGeometryFromAnchor(meshAnchor: ARMeshAnchor) -> SCNGeometry 
    let meshGeometry = meshAnchor.geometry
    let vertices = meshGeometry.vertices
    let normals = meshGeometry.normals
    let faces = meshGeometry.faces
    
    // use the MTL buffer that ARKit gives us
    let vertexSource = SCNGeometrySource(buffer: vertices.buffer, vertexFormat: vertices.format, semantic: .vertex, vertexCount: vertices.count, dataOffset: vertices.offset, dataStride: vertices.stride)
    
    let normalsSource = SCNGeometrySource(buffer: normals.buffer, vertexFormat: normals.format, semantic: .normal, vertexCount: normals.count, dataOffset: normals.offset, dataStride: normals.stride)
    // Copy bytes as we may use them later
    let faceData = Data(bytes: faces.buffer.contents(), count: faces.buffer.length)
    
    // create the geometry element
    let geometryElement = SCNGeometryElement(data: faceData, primitiveType: toSCNGeometryPrimitiveType(faces.primitiveType), primitiveCount: faces.count, bytesPerIndex: faces.bytesPerIndex)
    
    return SCNGeometry(sources: [vertexSource, normalsSource], elements: [geometryElement])


func toSCNGeometryPrimitiveType(_ ar: ARGeometryPrimitiveType) -> SCNGeometryPrimitiveType 
    switch ar 
    case .triangle: return .triangles
    default: fatalError("Unknown type")
    

最后，只要在 ARSCNViewDelegate renderer didUpdate: 函数中重构的几何图形发生变化，就更新场景节点：

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) 
    guard let meshAnchor = anchor as? ARMeshAnchor else 
        return
    
    
    let geometry = SCNGeometry.fromAnchor(meshAnchor: meshAnchor, setColors: false)
    geometry.firstMaterial?.colorBufferWriteMask = []

    node.geometry = geometry

    node.physicsBody!.physicsShape = SCNPhysicsShape(geometry: geometry, options: [.type: SCNPhysicsShape.ShapeType.concavePolyhedron])

您在 SceneKit 中创建的任何物理对象现在都应该能够与重建的场景进行交互：

【讨论】：

你在哪里声明到SCNGeometryPrimitiveType ？

这里有同样的问题。 toSCNGeometryPrimitiveType 在哪里声明？

我添加了一个基本的 impl 它，但您可能只需硬编码 SCNGeometryPrimitiveType.triangles。