Algorithms for 3D Shape Scanning with a Depth Camera

We describe a method for 3D object scanning by aligning depth scans that were taken from around an object with a Time-of-Flight (ToF) camera. These ToF cameras can measure depth scans at video rate. Due to comparably simple technology, they bear potential for economical production in big volumes. Our easy-to-use, cost-effective scanning solution, which is based on such a sensor, could make 3D scanning technology more accessible to everyday users. The algorithmic challenge we face is that the sensor's level of random noise is substantial and there is a nontrivial systematic bias. In this paper, we show the surprising result that 3D scans of reasonable quality can also be obtained with a sensor of such low data quality. Established filtering and scan alignment techniques from the literature fail to achieve this goal. In contrast, our algorithm is based on a new combination of a 3D superresolution method with a probabilistic scan alignment approach that explicitly takes into account the sensor's noise characteristics.

[1]  Reinhard Koch,et al.  Time-of-Flight sensor calibration for accurate range sensing , 2010, Comput. Vis. Image Underst..

[2]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[5]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[6]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[7]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[8]  Tim Weyrich,et al.  Capturing Time-of-Flight data with confidence , 2011, CVPR 2011.

[9]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[10]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[11]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[12]  Helmut Pottmann,et al.  Reassembling fractured objects by geometric matching , 2006, ACM Trans. Graph..

[13]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[14]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[16]  Yasuyuki Matsushita,et al.  High-quality shape from multi-view stereo and shading under general illumination , 2011, CVPR 2011.

[17]  Francis Schmitt,et al.  A Solution for the Registration of Multiple 3D Point Sets Using Unit Quaternions , 1998, ECCV.

[18]  Bernd Jähne,et al.  Radiometric and Spectrometric Calibrations, and Distance Noise Measurement of ToF Cameras , 2009, Dyn3D.

[19]  Pietro Perona,et al.  3D photography on your desk , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[20]  Young Min Kim,et al.  Design and calibration of a multi-view TOF sensor fusion system , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[21]  Reinhard Koch,et al.  Time-of-Flight Sensors in Computer Graphics , 2009, Eurographics.

[22]  Szymon Rusinkiewicz,et al.  Global non-rigid alignment of 3-D scans , 2007, ACM Trans. Graph..

[23]  Gerhard Rigoll,et al.  Resolution Enhancement of PMD Range Maps , 2008, DAGM-Symposium.

[24]  Rachid Deriche,et al.  Dense Depth Map Reconstruction: A Minimization and Regularization Approach which Preserves Discontinuities , 1996, ECCV.

[25]  Carlos Hernández,et al.  Practical 3D Reconstruction Based on Photometric Stereo , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[26]  Ira Kemelmacher-Shlizerman,et al.  Photometric Stereo with General, Unknown Lighting , 2006, International Journal of Computer Vision.

[27]  Bodo Rosenhahn,et al.  Pose estimation revisited , 2006 .

[28]  Dietmar Hildenbrand,et al.  Pose estimation based on Geometric Algebra , 2009 .

[29]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Wolfgang Straßer,et al.  Bayesian Point Cloud Reconstruction , 2006, Comput. Graph. Forum.

[31]  Hans Hagen,et al.  Visualization and Processing of Tensor Fields , 2014 .

[32]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[34]  Robert Bergevin,et al.  Towards a General Multi-View Registration Technique , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Marc Alexa,et al.  Depth Imaging by Combining Time-of-Flight and On-Demand Stereo , 2009, Dyn3D.

[36]  Sebastian Thrun,et al.  A Bayesian method for probable surface reconstruction and decimation , 2006, TOGS.

[37]  Reinhard Koch,et al.  A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images , 2007, DAGM-Symposium.

[38]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[39]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Alonzo Kelly,et al.  Experimental Characterization of Commercial Flash Ladar Devices , 2005 .

[42]  Didier Stricker,et al.  3D shape scanning with a Kinect , 2011, SIGGRAPH '11.

[43]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).