OpenCV：Surface Matching 3D

Posted 2022-06-07 wishchin

文档链接：https://docs.opencv.org/master/d9/d25/group__surface__matching.html

OpenCV在逐渐的吃PCL的地盘，不过工具还不够丰富。

此函数竟然使用点对特征和hash！

函数使用的ICP方法：Picky ICP方法；https://www.mathworks.com/matlabcentral/fileexchange/47152-icp-registration-using-efficient-variants-and-multi-resolution-scheme

Note about the License and Patents

The following patents have been issued for methods embodied in this software: "Recognition and pose determination of 3D objects in 3D scenes using geometric point pair descriptors and the generalized Hough Transform", Bertram Heinrich Drost, Markus Ulrich, EP Patent 2385483 (Nov. 21, 2012), assignee: MVTec Software GmbH, 81675 Muenchen (Germany); "Recognition and pose determination of 3D objects in 3D scenes", Bertram Heinrich Drost, Markus Ulrich, US Patent 8830229 (Sept. 9, 2014), assignee: MVTec Software GmbH, 81675 Muenchen (Germany). Further patents are pending. For further details, contact MVTec Software GmbH ([e-mail]).

Note that restrictions imposed by these patents (and possibly others) exist independently of and may be in conflict with the freedoms granted in this license, which refers to copyright of the program, not patents for any methods that it implements. Both copyright and patent law must be obeyed to legally use and redistribute this program and it is not the purpose of this license to induce you to infringe any patents or other property right claims or to contest validity of any such claims. If you redistribute or use the program, then this license merely protects you from committing copyright infringement. It does not protect you from committing patent infringement. So, before you do anything with this program, make sure that you have permission to do so not merely in terms of copyright, but also in terms of patent law.

Please note that this license is not to be understood as a guarantee either. If you use the program according to this license, but in conflict with patent law, it does not mean that the licensor will refund you for any losses that you incur if you are sued for your patent infringement.

Introduction to Surface Matching

Cameras and similar devices with the capability of sensation of 3D structure are becoming more common. Thus, using depth and intensity information for matching 3D objects (or parts) are of crucial importance for computer vision. Applications range from industrial control to guiding everyday actions for visually impaired people. The task in recognition and pose estimation in range images aims to identify and localize a queried 3D free-form object by matching it to the acquired database.

From an industrial perspective, enabling robots to automatically locate and pick up randomly placed and oriented objects from a bin is an important challenge in factory automation, replacing tedious and heavy manual labor. A system should be able to recognize and locate objects with a predefined shape and estimate the position with the precision necessary for a gripping robot to pick it up. This is where vision guided robotics takes the stage. Similar tools are also capable of guiding robots (and even people) through unstructured environments, leading to automated navigation. These properties make 3D matching from point clouds a ubiquitous necessity. Within this context, I will now describe the OpenCV implementation of a 3D object recognition and pose estimation algorithm using 3D features.

Surface Matching Algorithm Through 3D Features

The state of the algorithms in order to achieve the task 3D matching is heavily based on [52], which is one of the first and main practical methods presented in this area. The approach is composed of extracting 3D feature points randomly from depth images or generic point clouds, indexing them and later in runtime querying them efficiently. Only the 3D structure is considered, and a trivial hash table is used for feature queries.

While being fully aware that utilization of the nice CAD model structure in order to achieve a smart point sampling, I will be leaving that aside now in order to respect the generalizability of the methods (Typically for such algorithms training on a CAD model is not needed, and a point cloud would be sufficient). Below is the outline of the entire algorithm:

As explained, the algorithm relies on the extraction and indexing of point pair features, which are defined as follows:

F(m1,m2)=(||d||2,<(n1,d),<(n2,d),<(n1,n2))

where m1 and m2 are feature two selected points on the model (or scene), d is the difference vector, n1 and n2 are the normals at m1 and m2.

 

调用方法：

// pc is the loaded point cloud of the model
// (Nx6) and pcTest is a loaded point cloud of
// the scene (Mx6)
ppf_match_3d::PPF3DDetector detector(0.03, 0.05);
detector.trainModel(pc);
vector<Pose3DPtr> results;
detector.match(pcTest, results, 1.0/10.0, 0.05);
cout << "Poses: " << endl;
// print the poses
for (size_t i=0; i<results.size(); i++)

    Pose3DPtr pose = results[i];
    cout << "Pose Result " << i << endl;
    pose->printPose();