Deformable 3D registration with PWC-net optical flow and textured node correspondences

In this paper, we present an approach for the deformable registration of 3D data via an RGB-D camera to reduce depth distortions in featureless regions. We employ the established PWC-Net based Optical Flow algorithm to identify pixel correspondence between nearby frames and then densely and uniformly select transformation nodes. Color correspondence of the transformation nodes is used in both global and local deformations. Several experimental results show that the proposed method results in low distortion during the non-rigid registration of multiple RGB-D images.

[1]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Pushmeet Kohli,et al.  Fusion4D , 2016, ACM Trans. Graph..

[4]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[5]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ruigang Yang,et al.  Dynamic Non-Rigid Objects Reconstruction with a Single RGB-D Sensor , 2018, Sensors.

[7]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[8]  Rama Chellappa,et al.  Accuracy vs Efficiency Trade-offs in Optical Flow Algorithms , 1996, Comput. Vis. Image Underst..

[9]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[10]  Jan Kautz,et al.  PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Xiaoyang Liu,et al.  Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera , 2017, ACM Trans. Graph..

[12]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Szymon Rusinkiewicz,et al.  Global non-rigid alignment of 3-D scans , 2007, ACM Trans. Graph..

[14]  Ira Kemelmacher-Shlizerman,et al.  Total Moving Face Reconstruction , 2014, ECCV.

[15]  Leonidas J. Guibas,et al.  Eurographics Symposium on Geometry Processing (2007) Reconstruction of Deforming Geometry from Time-varying Point Clouds , 2022 .

[16]  Shahram Izadi,et al.  Motion2fusion , 2017, ACM Trans. Graph..

[17]  Bin Liang,et al.  An Iterative Closest Points Algorithm for Registration of 3D Laser Scanner Point Clouds with Geometric Features , 2017, Sensors.

[18]  Luc Vincent,et al.  Morphological grayscale reconstruction in image analysis: applications and efficient algorithms , 1993, IEEE Trans. Image Process..

[19]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[20]  P. Maragos.  3.3 – Morphological Filtering for Image Enhancement and Feature Detection , 2004 .