Geometrical 3D reconstruction using real-time RGB-D cameras

A RGB-D image combines, for each pixel, the classical three color channels with a fourth channel providing depth information. Devices that produce RGB-D images in real time with a rather good resolution are currently available on the market. With this type of device, it is possible to acquire and to process, in real time, 3D textured information, paving the way for numerous applications in the field of computer imaging and vision. In this paper, we analyse the accuracy of a low cost system and we see how this kind of device and the RGB-D images it produces allow us to acquire 3D models of real objects. A first application is presented that combines multiple RGB-D images of a static scene, taken from different viewpoints, in order to reconstruct a complete 3D model of the scene. A second application combines on-the-fly RGB-D images coming from multiple devices, generating a 3D model where the problems of occlusions inherent in monocular observations are drastically reduced.

[1]  Craig Gotsman,et al.  Articulated Object Reconstruction and Markerless Motion Capture from Depth Video , 2008, Comput. Graph. Forum.

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Joachim Hertzberg,et al.  Three-dimensional mapping with time-of-flight cameras , 2009 .

[4]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[5]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Kurt Konolige,et al.  FrameSLAM: From Bundle Adjustment to Real-Time Visual Mapping , 2008, IEEE Transactions on Robotics.

[7]  Didier Stricker,et al.  3D shape scanning with a Kinect , 2011, SIGGRAPH '11.

[8]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[9]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[10]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[11]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time dynamic 3D surface reconstruction and interaction , 2011, SIGGRAPH '11.

[13]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[15]  Tomasz Malisiewicz,et al.  Registration of multiple range scans as a location recognition problem: hypothesis generation, refinement and verification , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[16]  Sebastian Thrun,et al.  High-quality scanning using time-of-flight depth superresolution , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[17]  Ioannis Stamos,et al.  Automated feature-based range registration of urban scenes of large scale , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Guido M. Cortelazzo,et al.  3D registration by textured spin-images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[19]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[20]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Sang Wook Lee,et al.  ICP Registration Using Invariant Features , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[23]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.