Kinect关节识别

Posted 2020-11-02 81192

基础：添加KinectManager 组件

1）局部关节获取（参考插件场景KinectOverlayDemo1）

要获取局部某一关节及其位置，添加脚本JointOverlayer即可，通过Tracked joint参数可以分别获取到相关关节位置以及坐标。

 

2）所有关节获取（参考插件场景KinectOverlayDemo2）

 获取所有关节，可用脚本SkeletonOverlayer

也可参考如下代码

   /// <summary>
    /// 通过index获取用户的关节，并在每个关节点放置cube
    /// </summary>
    /// <param name="index"></param>
    private void GetBodyJoints(int index)
    {
        if(GetDetected())
        {
            long userId = kinectManger.GetUserIdByIndex(index);

            int jointsCount = kinectManger.GetJointCount();

            if(!isJointsCreated)
            {
                for (int i = 0; i < jointsCount; i++)
                {
                    joints.Add(Instantiate(jointGo));
                }
                isJointsCreated = true;
            }
            else
            {
                for (int i = 0; i < jointsCount; i++)
                {
                   joints[i].transform.position = kinectManger.GetJointKinectPosition(userId,i);
                   joints[i].name = i.ToString();
                }
            } 
        }
    }

 

3）交互功能（参考插件场景KinectOverlayDemo3）

交互功能即为通过手势实现鼠标点击功能需要InteractionManager以及HandOverlayer脚本，通过HandOverlayer/InteractionManager脚本上的参数带texture的参数来实现抓取松开等样式

如果要把手的位置反应到UI上则需要特殊处理，因为我们获取的手的坐标单位为米，反应到unity中只是为数字（比如左手移动0.2m），但是如果分辨率为1920X1080，则手的移动0.2相对于分辨率来说几乎为零，所以需要通过UICamera.ViewportToWorldPoint来转化一下（或者   UICamera.ViewportToScreenPoint）下述脚本为在JointOverlayer中修改的update部分，通过获取手移动的box（范围），然后通过手实际位置转换成相对坐标，然后转化到实际分辨率下的位置。

    void Update () 
    {  
        KinectManager manager = KinectManager.Instance;
        
        if(manager && manager.IsInitialized() && foregroundCamera)
        {
            //backgroundImage.renderer.material.mainTexture = manager.GetUsersClrTex();
            if(backgroundImage && (backgroundImage.texture == null))
            {
                backgroundImage.texture = manager.GetUsersClrTex();
            }
            
            // get the background rectangle (use the portrait background, if available)
            Rect backgroundRect = foregroundCamera.pixelRect;
            PortraitBackground portraitBack = PortraitBackground.Instance;
            
            if(portraitBack && portraitBack.enabled)
            {
                backgroundRect = portraitBack.GetBackgroundRect();
            }

            // overlay the joint
            int iJointIndex = (int)trackedJoint;

            if (manager.IsUserDetected())
            {
                for (int i = 0; i < 6; i++)
                {
                    long userId = manager.GetUserIdByIndex(i);

                    //manager.IsUserTracked(userId);

                    if (manager.IsJointTracked(userId, iJointIndex))
                    {
                        //MainFunController.theController.OnDebug("Joint tracked\r\n" + GetPara(manager));

                        overlayObject.gameObject.SetActive(true);
                        Vector3 rightIboxRightBotBack = Vector3.zero, rightIboxRightTopFront = Vector3.zero, rightHandScreenPos = Vector3.zero;
                        bool isrightIboxValid = false;
                        isrightIboxValid = manager.GetRightHandInteractionBox(userId, ref rightIboxRightBotBack, ref rightIboxRightTopFront, isrightIboxValid);

                        if (isrightIboxValid && manager.GetJointTrackingState(userId, (int)KinectInterop.JointType.HandRight) != KinectInterop.TrackingState.NotTracked)
                        {
                            Vector3 rightHandPos = manager.GetJointPosition(userId, (int)KinectInterop.JointType.HandRight);

                            rightHandScreenPos.x = Mathf.Clamp01((rightHandPos.x - rightIboxRightBotBack.x) / (rightIboxRightTopFront.x - rightIboxRightBotBack.x));
                            rightHandScreenPos.y = Mathf.Clamp01((rightHandPos.y - rightIboxRightBotBack.y) / (rightIboxRightTopFront.y - rightIboxRightBotBack.y));
                            rightHandScreenPos.z = Mathf.Clamp01((rightIboxRightBotBack.z - rightHandPos.z) / (rightIboxRightBotBack.z - rightIboxRightTopFront.z));
                        }

                        Vector3 cursorTargetPos = UICamera.ViewportToWorldPoint(rightHandScreenPos);
                     
                        overlayObject.position = Vector2.Lerp(overlayObject.position, cursorTargetPos, 1f);
                        break;
                    }
                    else
                    {
                        overlayObject.gameObject.SetActive(false);
                        //MainFunController.theController.OnDebug("Joint miss");
                        //isFirstCatch = true;
                        //CollisionEvent.theEvent.OnStop();
                    }
                }
            }
            else
            {
                //debugText.text = "body miss";
            }

        }
    }

 

 

 

注：应用时建议复制此脚本，并修改名称，然后做自己的修改