3D scanning of cultural heritage with consumer depth cameras

Three dimensional reconstruction of cultural heritage objects is an expensive and time-consuming process. Recent consumer real-time depth acquisition devices, like Microsoft Kinect, allow very fast and simple acquisition of 3D views. However 3D scanning with such devices is a challenging task due to the limited accuracy and reliability of the acquired data. This paper introduces a 3D reconstruction pipeline suited to use consumer depth cameras as hand-held scanners for cultural heritage objects. Several new contributions have been made to achieve this result. They include an ad-hoc filtering scheme that exploits the model of the error on the acquired data and a novel algorithm for the extraction of salient points exploiting both depth and color data. Then the salient points are used within a modified version of the ICP algorithm that exploits both geometry and color distances to precisely align the views even when geometry information is not sufficient to constrain the registration. The proposed method, although applicable to generic scenes, has been tuned to the acquisition of sculptures and in this connection its performance is rather interesting as the experimental results indicate.

[1]  Nassir Navab,et al.  Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[3]  ZhouJin,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012 .

[4]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Guido M. Cortelazzo,et al.  Time-of-Flight Cameras and Microsoft Kinect™ , 2012, Springer Briefs in Electrical and Computer Engineering.

[6]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[7]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[8]  Fabio Remondino,et al.  Heritage Recording and 3D Modeling with Photogrammetry and 3D Scanning , 2011, Remote. Sens..

[9]  George Pavlidis,et al.  Methods for 3D digitization of Cultural Heritage , 2007 .

[10]  Reinhard Koch,et al.  Acquisition of detailed models for virtual reality , 2000 .

[11]  Sabry F. El-Hakim,et al.  Detailed 3D reconstruction of large-scale heritage sites with integrated techniques , 2004, IEEE Computer Graphics and Applications.

[12]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[13]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Kourosh Khoshelham,et al.  Accuracy analysis of kinect depth data , 2012 .

[15]  Dieter Fritsch,et al.  CO-REGISTRATION OF KINECT POINT CLOUDS BASED ON IMAGE AND OBJECT SPACE OBSERVATIONS , 2013 .

[16]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[17]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Andrea Torsello,et al.  Sampling Relevant Points for Surface Registration , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[19]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision: Research Topics and Applications , 2012 .

[20]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[21]  Didier Stricker,et al.  Algorithms for 3D Shape Scanning with a Depth Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  John J. Leonard,et al.  Robust Tracking for Real-Time Dense RGB-D Mapping with Kintinuous , 2012 .

[23]  Guido M. Cortelazzo,et al.  A Probabilistic Approach to ToF and Stereo Data Fusion , 2010 .

[24]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision , 2013, Advances in Computer Vision and Pattern Recognition.

[26]  Joachim Hertzberg,et al.  An Explicit Loop Closing Technique for 6D SLAM , 2009, ECMR.

[27]  Holly E. Rushmeier,et al.  The 3D Model Acquisition Pipeline , 2002, Comput. Graph. Forum.

[28]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[29]  Guido M. Cortelazzo,et al.  Automatic 3D modeling of textured cultural heritage objects , 2004, IEEE Transactions on Image Processing.

[30]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[31]  Juho Kannala,et al.  Joint Depth and Color Camera Calibration with Distortion Correction , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Guido M. Cortelazzo,et al.  Probabilistic ToF and Stereo Data Fusion Based on Mixed Pixels Measurement Models , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  R. Sablatnig,et al.  ANCIENT COINS & CERAMICS-3 D AND 2 D DOCUMENTATION FOR PRESERVATION AND RETRIEVAL OF LOST HERITAGE , 2007 .

[35]  Guido M. Cortelazzo,et al.  Handheld scanning with 3D cameras , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).