Robust shape from depth images with GR2T

Extension of GRRT inferential framework with nuisance parameters.3D shape inference from multiple view depth images.Estimation of extrinsic camera parameters.3D shape reconstruction. This paper proposes to infer accurately a 3D shape of an object captured by a depth camera from multiple view points. The Generalised Relaxed Radon Transform (GR2T) 1 is used here to merge all depth images in a robust kernel density estimate that models the surface of an object in the 3D space. The kernel is tailored to capture the uncertainty associated with each pixel in the depth images. The resulting cost function is suitable for stochastic exploration with gradient ascent algorithms when the noise of the observations is modelled with a differentiable distribution. When merging several depth images captured from several view points, extrinsic camera parameters need to be known accurately, and we extend GR2T to also estimate these nuisance parameters. We illustrate qualitatively the performance of our modelling and we assess quantitatively the accuracy of our 3D shape reconstructions computed from depth images captured with a Kinect camera.

[1]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[2]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[3]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[4]  Jongmoo Choi,et al.  Laser scan quality 3-D face modeling using a low-cost depth camera , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[5]  Mark Pauly,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[6]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[7]  Roberto Scopigno,et al.  Fully Automatic Registration of Image Sets on Approximate Geometry , 2012, International Journal of Computer Vision.

[8]  Pavel Krsek,et al.  Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm , 2005, Image Vis. Comput..

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Ming Zeng,et al.  A memory-efficient kinectfusion using octree , 2012, CVM'12.

[12]  Yin Chen,et al.  Personalized avatar capture using two Kinects in a moment , 2012, SA '12.

[13]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[14]  Sang Wook Lee,et al.  Maximum-Likelihood Registration of Range Images with Missing Data , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[16]  Marc Pollefeys,et al.  Practical Methods for Convex Multi-view Reconstruction , 2010, ECCV.

[17]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[18]  F. Sommen,et al.  The general quadratic Radon transform , 1998 .

[19]  Rozenn Dahyot,et al.  Generalised relaxed Radon transform (GR2T) for robust inference , 2013, Pattern Recognit..

[20]  David W. Scott,et al.  Parametric Statistical Modeling by Minimum Integrated Square Error , 2001, Technometrics.

[21]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[22]  Peter Meer,et al.  Generalized projection based M-estimator: Theory and applications , 2011, CVPR 2011.

[23]  Rozenn Dahyot,et al.  Bayesian 3D shape from silhouettes , 2013, Digit. Signal Process..

[24]  Balaji Vasan Srinivasan.  GPUML : Graphical processors for speeding up kernel machines , 2010 .

[25]  Ming Zeng,et al.  Octree-based fusion for realtime 3D reconstruction , 2013, Graph. Model..

[26]  Marsette Vona,et al.  Moving Volume KinectFusion , 2012, BMVC.

[27]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[29]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[30]  Andrea Censi,et al.  An ICP variant using a point-to-line metric , 2008, 2008 IEEE International Conference on Robotics and Automation.

[31]  John Thompson,et al.  European Signal Processing Conference (EUSIPCO) , 2016 .

[32]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[33]  R. Dahyot,et al.  Smooth Kernel Density Estimate for Multiple View Reconstruction , 2010, 2010 Conference on Visual Media Production.

[34]  Sebastian Thrun,et al.  Unsupervised Intrinsic Calibration of Depth Sensors via SLAM , 2013, Robotics: Science and Systems.

[35]  Peter Meer,et al.  Generalized Projection-Based M-Estimator , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[37]  Didier Stricker,et al.  Algorithms for 3D Shape Scanning with a Depth Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Guannan Gao,et al.  Probabilistic Hough Transform , 2011 .

[39]  Josef Kittler,et al.  A formal definition of the Hough transform: Properties and relationships , 2004, Journal of Mathematical Imaging and Vision.

[40]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Rozenn Dahyot,et al.  Extrinsic camera parameters estimation for shape-from-depths , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[42]  H. Seidel,et al.  Pattern-aware Deformation Using Sliding Dockers , 2011, SIGGRAPH 2011.

[43]  Juho Kannala,et al.  Joint Depth and Color Camera Calibration with Distortion Correction , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  John J. Leonard,et al.  Robust real-time visual odometry for dense RGB-D mapping , 2013, 2013 IEEE International Conference on Robotics and Automation.

[46]  Jonathan Ruttle,et al.  Statistical framework for multi sensor fusion and 3D reconstruction , 2012 .

[47]  Lei Zhang,et al.  Robust ICP Registration Using Biunique Correspondence , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[48]  Clark F. Olson,et al.  Constrained Hough Transforms for Curve Detection , 1999, Comput. Vis. Image Underst..