3D reconstruction methods for digital preservation of cultural heritage: A survey

A overview of the state-of-the-art approaches for 3D reconstruction.We focus this overview on cultural heritage preservation.We point open problems and difficulties on this field. 3D reconstruction, refers to capturing and reproducing the shape and appearance of an arbitrary object or scene given depth and color information. This is a broad research area within the computer vision field involving many stages and still open problems. The digital preservation of cultural heritage is a specially challenging application of 3D reconstruction. Cultural heritage objects and sites greatly differ from each other and a maximized fidelity of the 3D reconstruction is a core requirement. The literature on this topic has substantially increased in the past years, mostly due to the variety of scenarios and the development of new depth sensing devices as well as techniques able to deal with this issue. In our search to develop a complete 3D reconstruction pipeline, we have comprehensively studied techniques related to this topic and divided the 3D digitization process in four major overviews: image acquisition, view registration, mesh integration and texture generation. We present the state-of-the-art approaches and challenges of each stage.

[1]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[2]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[3]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[4]  François Blais.  Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[5]  Sunghee Choi,et al.  The power crust, unions of balls, and the medial axis transform , 2001, Comput. Geom..

[6]  Ross T. Whitaker,et al.  A Level-Set Approach to 3D Reconstruction from Range Data , 1998, International Journal of Computer Vision.

[7]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Ryo Kurazume,et al.  Fast alignment of 3D geometrical models and 2D color images using 2D distance maps , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[9]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[10]  Marc Pollefeys,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, SIGGRAPH 2008.

[11]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Hans-Peter Seidel,et al.  Multi-level partition of unity implicits , 2005, SIGGRAPH Courses.

[13]  Claudio Rocchini,et al.  Three-dimensional modelling of statues: the Minerva of Arezzo , 2002 .

[14]  Marco Callieri,et al.  Innovative uses of 3D digital technologies to assist the restoration of a fragmented terracotta statue , 2013 .

[15]  Luciano Silva,et al.  3D preserving xviii century barroque masterpiece: Challenges and results on the digital preservation of Aleijadinho’s sculpture of the Prophet Joel , 2012 .

[16]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[17]  Paolo Cignoni,et al.  Exploiting the scanning sequence for automatic registration of large sets of range maps , 2005, Comput. Graph. Forum.

[18]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Randal C. Burns,et al.  Parallel Poisson Surface Reconstruction , 2009, ISVC.

[20]  Yi-Ping Hung,et al.  RANSAC-Based DARCES: A New Approach to Fast Automatic Registration of Partially Overlapping Range Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Tamal K. Dey,et al.  Tight cocone: a water-tight surface reconstructor , 2003, SM '03.

[22]  Leif Kobbelt,et al.  Robust reconstruction of watertight 3D models from non-uniformly sampled point clouds without normal information , 2006, SGP '06.

[23]  Ken Brodlie,et al.  Improving the Robustness and Accuracy of the Marching Cubes Algorithm for Isosurfacing , 2003, IEEE Trans. Vis. Comput. Graph..

[24]  Kari Pulli,et al.  Multiview registration for large data sets , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[25]  Tamal K. Dey,et al.  Delaunay based shape reconstruction from large data , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[26]  Yoav Y. Schechner,et al.  Depth from Defocus vs. Stereo: How Different Really Are They? , 2004, International Journal of Computer Vision.

[27]  Takeshi Masuda,et al.  Object shape modelling from multiple range images by matching signed distance fields , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[28]  Szymon Rusinkiewicz,et al.  Global non-rigid alignment of 3-D scans , 2007, SIGGRAPH 2007.

[29]  Nina Amenta,et al.  Laser Scanner Super-resolution , 2006, PBG@SIGGRAPH.

[30]  William G. Rippey,et al.  An Integrated Vision Touch-Probe System for Dimensional Inspection Tasks , 1995 .

[31]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Bruno Lévy,et al.  Mesh parameterization: theory and practice , 2007, SIGGRAPH Courses.

[33]  Katsushi Ikeuchi,et al.  Adaptively merging large-scale range data with reflectance properties , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Katsushi Ikeuchi,et al.  Taking consensus of signed distance field for complementing unobservable surface , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[35]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[36]  Meenakshisundaram Gopi,et al.  Surface Reconstruction based on Lower Dimensional Localized Delaunay Triangulation , 2000, Comput. Graph. Forum.

[37]  Paolo Cignoni,et al.  Pushing time-of-flight scanners to the limit , 2009, VAST'09.

[38]  Maurício Pamplona Segundo,et al.  Automating 3D reconstruction pipeline by surf-based alignment , 2012, 2012 19th IEEE International Conference on Image Processing.

[39]  Gabriel Taubin,et al.  3D Photography , 2015 .

[40]  Luiz Velho,et al.  Geometry Super-Resolution by Example , 2009, 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing.

[41]  Paolo Cignoni,et al.  A low cost 3D scanner based on structured light , 2001 .

[42]  Marc Rioux,et al.  Active Optical 3D Imaging for Heritage Applications , 2002, IEEE Computer Graphics and Applications.

[43]  H. Pottmann,et al.  Reassembling fractured objects by geometric matching , 2006, SIGGRAPH 2006.

[44]  Paul J. Besl,et al.  Surfaces in Range Image Understanding , 1988, Springer Series in Perception Engineering.

[45]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Robert Bergevin,et al.  Towards a General Multi-View Registration Technique , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Luciano Silva,et al.  A 3D reconstruction pipeline for digital preservation , 2009, CVPR.

[48]  Adrian Hilton,et al.  Registration of multiple point sets , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[49]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[50]  A. Grün,et al.  High definition 3D-scanning of arts objects and paintings , 2007 .

[51]  Takeshi Masuda,et al.  Generation of geometric model by registration and integration of multiple range images , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[52]  Pedro V. Sander,et al.  Signal-specialized parameterization for piecewise linear reconstruction , 2004, SGP '04.

[53]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[54]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[55]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[56]  Yücel Yemez,et al.  A volumetric fusion technique for surface reconstruction from silhouettes and range data , 2007, Comput. Vis. Image Underst..

[57]  Alla Sheffer,et al.  Mesh parameterization: theory and practice Video files associated with this course are available from the citation page , 2007, SIGGRAPH Courses.

[58]  Sunghee Choi,et al.  A simple algorithm for homeomorphic surface reconstruction , 2000, SCG '00.

[59]  Gabriel Taubin,et al.  Building a Digital Model of Michelangelo's Florentine Pietà , 2002, IEEE Computer Graphics and Applications.

[60]  Luciano Silva,et al.  3D reconstruction of cultural heritages: Challenges and advances on precise mesh integration , 2012, Comput. Vis. Image Underst..

[61]  Katsushi Ikeuchi,et al.  Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute , 2002 .

[62]  Sunghee Choi,et al.  The power crust , 2001, SMA '01.

[63]  In-So Kweon,et al.  Capturing Village-level Heritages with a Hand-held Camera-Laser Fusion Sensor , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[64]  Martin Kampel,et al.  Combining shape from silhouette and shape from structured light for volume estimation of archaeological vessels , 2002, Object recognition supported by user interaction for service robots.

[65]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[66]  Holly E. Rushmeier,et al.  The 3D Model Acquisition Pipeline , 2002, Comput. Graph. Forum.

[67]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[68]  Sabry F. El-Hakim,et al.  Virtualizing a Byzantine Crypt by Combining High-resolution Textures with Laser Scanner 3D Data , 2002 .

[69]  Andrew W. Fitzgibbon,et al.  Simultaneous Registration of Multiple Range Views for Use in Reverse Engineering of CAD Models , 1998, Comput. Vis. Image Underst..

[70]  David K. Wehe,et al.  Multiview registration of 3D scenes by minimizing error between coordinate frames , 2004 .

[71]  Paolo Cignoni,et al.  Minimizing user intervention in registering 2D images to 3D models , 2005, The Visual Computer.

[72]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[73]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[74]  Herbert Edelsbrunner,et al.  Shape Reconstruction with Delaunay Complex , 1998, LATIN.

[75]  Steven J. Gortler,et al.  Geometry images , 2002, SIGGRAPH.

[76]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[77]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[78]  George Wolberg,et al.  Multiview Geometry for Texture Mapping 2D Images Onto 3D Range Data , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[79]  Hong Zhou,et al.  Accurate integration of multi-view range images using k-means clustering , 2008, Pattern Recognit..

[80]  Hubert Mara,et al.  GigaMesh and Gilgamesh ? 3D Multiscale Integral Invariant Cuneiform Character Extraction , 2010, VAST.

[81]  Ioannis Stamos,et al.  3-D model construction using range and image data , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[82]  Zhang Liyan,et al.  Segmentation and parametrization of arbitrary polygon meshes , 2004, Geometric Modeling and Processing, 2004. Proceedings.

[83]  Massimiliano Corsini,et al.  Documenting and Monitoring Small Fractures on Michelangelo's David , 2010 .

[84]  Ioannis Stamos,et al.  The Beauvais Cathedral Project , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[85]  John C. Hart,et al.  Seamster: inconspicuous low-distortion texture seam layout , 2002, IEEE Visualization, 2002. VIS 2002..

[86]  C. Q. Little,et al.  Registration of range data using a hybrid simulated annealing and iterative closest point algorithm , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[87]  Pedro V. Sander,et al.  Signal-Specialized Parametrization , 2002, Rendering Techniques.

[88]  S. Osher,et al.  Fast surface reconstruction using the level set method , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[89]  Adrian Hilton,et al.  Reliable Surface Reconstructiuon from Multiple Range Images , 1996, ECCV.

[90]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[91]  Alessandro Rizzi,et al.  Reality-based 3D documentation of natural and cultural heritage sites—techniques, problems, and examples , 2010 .

[92]  Katsushi Ikeuchi,et al.  Robust and adaptive integration of multiple range images with photometric attributes , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[93]  N. Mitra,et al.  4-points congruent sets for robust pairwise surface registration , 2008, SIGGRAPH 2008.

[94]  James F. O'Brien,et al.  Spectral surface reconstruction from noisy point clouds , 2004, SGP '04.

[95]  Denis Laurendeau,et al.  A General Surface Approach to the Integration of a Set of Range Views , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[96]  Richard J. Radke,et al.  Filling large holes in LiDAR data by inpainting depth gradients , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[97]  Demetri Terzopoulos,et al.  Constraints on Deformable Models: Recovering 3D Shape and Nonrigid Motion , 1988, Artif. Intell..

[98]  Luciano Silva,et al.  Digital preservation of Brazilian indigenous artworks: Generating high quality textures for 3D models , 2012 .

[99]  Katsushi Ikeuchi,et al.  Shape representation and image segmentation using deformable surfaces , 1992, Image Vis. Comput..

[100]  Kim L. Boyer,et al.  Multiview range image registration using the surface interpenetration measure , 2007, Image Vis. Comput..

[101]  Gabriel Taubin,et al.  The ball-pivoting algorithm for surface reconstruction , 1999, IEEE Transactions on Visualization and Computer Graphics.

[102]  George Pavlidis,et al.  Methods for 3D digitization of Cultural Heritage , 2007 .

[103]  Daniel Cohen-Or,et al.  Competing Fronts for Coarse–to–Fine Surface Reconstruction , 2006, Comput. Graph. Forum.

[104]  Antonio Brogi,et al.  Digital reconstruction of the Arrigo VII funerary complex , 2004, VAST.

[105]  Leonidas J. Guibas,et al.  Robust global registration , 2005, SGP '05.

[106]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[107]  Michael Potmesil.  Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[108]  Bruno Lévy,et al.  Least squares conformal maps for automatic texture atlas generation , 2002, ACM Trans. Graph..

[109]  Randal C. Burns,et al.  Multilevel streaming for out-of-core surface reconstruction , 2007, Symposium on Geometry Processing.

[110]  Holly Rushmeier,et al.  Eternal Egypt: Experiences and Research Directions , 2006 .

[111]  Avinash C. Kak,et al.  3D Modeling of Optically Challenging Objects , 2008, IEEE Transactions on Visualization and Computer Graphics.

[112]  Jindong Chen,et al.  Automatic Reconstruction of 3D CAD Models from Digital Scans , 1999, Int. J. Comput. Geom. Appl..

[113]  Paolo Cignoni,et al.  The Marching Intersections algorithm for merging range images , 2003, The Visual Computer.

[114]  Atsushi Nakazawa,et al.  The Great Buddha Project: Digitally Archiving, Restoring, and Analyzing Cultural Heritage Objects , 2007, International Journal of Computer Vision.

[115]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[116]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[117]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[118]  Charlie C. L. Wang,et al.  Reduce the stretch in surface flattening by finding cutting paths to the surface boundary , 2004, Comput. Aided Des..

[119]  Jens Guehring,et al.  Dense 3D surface acquisition by structured light using off-the-shelf components , 2000, IS&T/SPIE Electronic Imaging.

[120]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[121]  Hisham El-Shishiny,et al.  Scanning and processing 3D objects for Web display , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[122]  Katsushi Ikeuchi,et al.  Hole Filling of a 3D Model by Flipping Signs of a Signed Distance Field in Adaptive Resolution , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[123]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[124]  Tong Lee,et al.  Surface registration using a dynamic genetic algorithm , 2004, Pattern Recognit..

[125]  D. Cohen-Or,et al.  Robust moving least-squares fitting with sharp features , 2005, ACM Trans. Graph..

[126]  Martin D. Levine,et al.  Registering Multiview Range Data to Create 3D Computer Objects , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[127]  Katsushi Ikeuchi,et al.  Consensus surfaces for modeling 3D objects from multiple range images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[128]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[129]  Joseph Shamir,et al.  Range Imaging With Adaptive Color Structured Light , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[130]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[131]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[132]  Kim L. Boyer,et al.  Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.