Algorithm for Three-Dimensional Reconstruction of Nonrigid Objects Using a Depth Camera

Abstract —An algorithm for 3D reconstruction of objects with nonrigid shape using an RGB-D depth camera is proposed. The algorithm can be used in medicine, agriculture, robotics, virtual reality, and human–computer interaction. The proposed algorithm makes it possible to accurately reconstruct a 3D object with one depth camera without restricting camera movement and without using a priori information about an object shape. The reconstruction process consists of the following steps: input of information using an RGB-D camera, registration with a modified iterative closest point algorithm, and dynamic construction of a dense 3D model of objects. The efficiency of the proposed algorithm is evaluated using experimental data and is compared with the modern methods of registration. The results show that the proposed algorithm can accurately reconstruct 3D nonrigid objects on complex scenes with one depth camera.

[1]  Vitaly Kober,et al.  3D face recognition based on matching of facial surfaces , 2015, SPIE Optical Engineering + Applications.

[2]  Vitaly Kober,et al.  A robust HOG-based descriptor for pattern recognition , 2016, Optical Engineering + Applications.

[3]  Vitaly Kober,et al.  Face recognition based on matching of local features on 3D dynamic range sequences , 2016, Optical Engineering + Applications.

[4]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Nico Blodow,et al.  General 3D modelling of novel objects from a single view , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Jiaolong Yang,et al.  Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  V. N. Karnaukhov,et al.  A New Invariant to Illumination Feature Descriptor for Pattern Recognition , 2018, Journal of Communications Technology and Electronics.

[8]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[9]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[10]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  V. Kober,et al.  A non‐iterative method for approximation of the exact solution to the point‐to‐plane variational problem for orthogonal transformations , 2018, Mathematical Methods in the Applied Sciences.

[12]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[13]  Adel Hafiane,et al.  3D Reconstruction of the proximal femur shape from few pairs of x-ray radiographs , 2017, Signal Process. Image Commun..

[14]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[15]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[16]  Vitaly Kober,et al.  Accurate generation of the 3D map of environment with a RGB-D camera , 2017, Optical Engineering + Applications.

[17]  Vitaly Kober,et al.  Accurate alignment of RGB-D frames for 3D map generation , 2018, Optical Engineering + Applications.

[18]  Vitaly Kober,et al.  A point-to-plane registration algorithm for orthogonal transformations , 2018, Optical Engineering + Applications.

[19]  W. Kabsch.  A solution for the best rotation to relate two sets of vectors , 1976 .

[20]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Gabriel Taubin,et al.  The ball-pivoting algorithm for surface reconstruction , 1999, IEEE Transactions on Visualization and Computer Graphics.

[22]  Vitaly Kober,et al.  Adaptive algorithm for the SLAM design with a RGB-D camera , 2019, Optical Engineering + Applications.

[23]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Mohamed Atri,et al.  Robust technique for 3D shape reconstruction , 2017, J. Comput. Sci..

[25]  Vitaly Kober,et al.  Conformal Parameterization and Curvature Analysis for 3D Facial Recognition , 2015, 2015 International Conference on Computational Science and Computational Intelligence (CSCI).

[26]  Yu Zhong,et al.  Intrinsic shape signatures: A shape descriptor for 3D object recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[27]  V. V. Kuznetsov,et al.  A method of face recognition using 3D facial surfaces , 2017 .

[28]  Cordelia Schmid,et al.  A sparse texture representation using local affine regions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  David S. Doermann,et al.  Robust point matching for nonrigid shapes by preserving local neighborhood structures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Nico Blodow,et al.  Persistent Point Feature Histograms for 3D Point Clouds , 2008 .

[31]  Stephen M. Smith,et al.  SUSAN—A New Approach to Low Level Image Processing , 1997, International Journal of Computer Vision.

[32]  Julia Diaz-Escobar,et al.  LUIFT: LUminance Invariant Feature Transform , 2018, Mathematical Problems in Engineering.

[33]  V. N. Karnaukhov,et al.  Descriptor-based tracking algorithm using a depth camera , 2017 .

[34]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Benjamin Bustos,et al.  Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes , 2011, The Visual Computer.

[36]  Vitaly Kober,et al.  Accurate three-dimensional pose recognition from monocular images using template matched filtering , 2016 .

[37]  Andrea Censi,et al.  An ICP variant using a point-to-line metric , 2008, 2008 IEEE International Conference on Robotics and Automation.

[38]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..