Online loop closure for real-time interactive 3D scanning

We present a real-time interactive 3D scanning system that allows users to scan complete object geometry by turning the object around in front of a real-time 3D range scanner. The incoming 3D surface patches are registered and integrated into an online 3D point cloud. In contrast to previous systems the online reconstructed 3D model also serves as final result. Registration error accumulation which leads to the well-known loop closure problem is addressed already during the scanning session by distorting the object as rigidly as possible. Scanning errors are removed by explicitly handling outliers based on visibility constraints. Thus, no additional post-processing is required which otherwise might lead to artifacts in the model reconstruction. Both geometry and texture are used for registration which allows for a wide range of objects with different geometric and photometric properties to be scanned. We show the results of our modeling approach on several difficult real-world objects. Qualitative and quantitative results are given for both synthetic and real data demonstrating the importance of online loop closure and outlier handling for model reconstruction. We show that our real-time scanning system has comparable accuracy to offline methods with the additional benefit of immediate feedback and results.

[1]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[2]  Yasushi Yagi,et al.  Dynamic scene shape reconstruction using a single structured light pattern , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Kari Pulli,et al.  Multiview registration for large data sets , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[4]  Jean-Daniel Deschênes,et al.  Interactive Modeling with Automatic Online Compression , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[5]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.

[6]  Luc Van Gool,et al.  Real-time connectivity constrained depth map computation using programmable graphics hardware , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[8]  Adrian Hilton,et al.  Geometric fusion for a hand-held 3D sensor , 2000 .

[9]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[10]  Tom Drummond,et al.  ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition , 2009, BMVC.

[11]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[12]  Holly E. Rushmeier,et al.  The 3D Model Acquisition Pipeline , 2002, Comput. Graph. Forum.

[13]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[14]  Marc Rioux,et al.  Three-dimensional registration using range and intensity information , 1994, Other Conferences.

[15]  Patrick J. Flynn,et al.  A Survey Of Free-Form Object Representation and Recognition Techniques , 2001, Comput. Vis. Image Underst..

[16]  Alice Biber,et al.  Time-of-flight range imaging with a custom solid state image sensor , 1999, Industrial Lasers and Inspection.

[17]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jean-Arcady Meyer,et al.  Real-time visual loop-closure detection , 2008, 2008 IEEE International Conference on Robotics and Automation.

[19]  Sang Wook Lee,et al.  Range data registration using photometric features , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Luc Van Gool,et al.  Online 3D acquisition and model integration , 2003 .

[21]  Sebastian Weik,et al.  Registration of 3-D partial surface models using luminance and depth information , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[22]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[23]  Sing Bing Kang,et al.  Registration and integration of textured 3-D data , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[24]  Jean-Daniel Deschênes,et al.  A unified representation for interactive 3D modeling , 2004 .

[25]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[26]  Luc Van Gool,et al.  In-hand scanning with online loop closure , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[27]  Song Zhang,et al.  High-Resolution, Real-time 3D Shape Acquisition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[28]  Heinz Hügli,et al.  Multi-feature matching algorithm for free-form 3D surface registration , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[29]  Adrian Hilton,et al.  Implicit Surface-Based Geometric Fusion , 1998, Comput. Vis. Image Underst..

[30]  K. Nechvíle.  The High Resolution , 2005 .

[31]  Nelson L. Max,et al.  Weights for Computing Vertex Normals from Facet Normals , 1999, J. Graphics, GPU, & Game Tools.

[32]  Leif Kobbelt,et al.  A Surface-Growing Approach to Multi-View Stereo Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Marc Levoy,et al.  Geometrically stable sampling for the ICP algorithm , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[34]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[35]  G. Iddan,et al.  3D IMAGING IN THE STUDIO (AND ELSEWHERE...) , 2001 .

[36]  Luc Van Gool,et al.  Accurate and robust registration for in-hand modeling , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Andrew Fitzgibbon,et al.  A Hand-held Optical Surface Scanner for Environmental Modeling and Virtual Reality , 1996 .

[38]  John B. Moore,et al.  Global registration of multiple 3D point sets via optimization-on-a-manifold , 2005, SGP '05.

[39]  M. Pauly,et al.  Embedded deformation for shape manipulation , 2007, SIGGRAPH 2007.

[40]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[41]  M. Gross,et al.  Algebraic point set surfaces , 2007, SIGGRAPH 2007.

[42]  Andrew E. Johnson,et al.  Registration and integration of textured 3-D data , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[43]  Guy Godin,et al.  Accurate 3D acquisition of freely moving objects , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[44]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[45]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[46]  Luc Van Gool,et al.  Adaptive Scanning for Online 3D Model Acquisition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[47]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[49]  Thierry Oggier,et al.  CCD / CMOS Lock-In Pixel for Range Imaging : Challenges , Limitations and State-ofthe-Art , 2005 .