Surface Reconstruction and Display from Range and Color Data

This paper addresses the problem of scanning both the color and geometry of real objects and displaying realistic images of the scanned objects from arbitrary viewpoints. We describe a complete system that uses a stereo camera setup with active lighting to scan the object surface geometry and color. Scans expressed in sensor coordinates are registered into a single object-centered coordinate system by aligning both the color and geometry where the scans overlap. The range data are integrated into a surface model using a robust hierarchical space carving method. The fit of the resulting approximate mesh to data is improved and the mesh structure is simplified using mesh optimization methods. In addition, a method for view-dependent texturing of the reconstructed surfaces is described. The method projects the color data from the input images onto the surface model and blends the various images depending on the location of the viewpoint and other factors such as surface orientation.

[1]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[2]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[3]  Leonidas J. Guibas,et al.  Primitives for the manipulation of general subdivisions and the computation of Voronoi diagrams , 1983, STOC.

[4]  Michael Potmesil,et al.  Generating Models of Solid Objects by Matching 3D Surface Segments , 1983, IJCAI.

[5]  Gene H. Golub,et al.  Matrix computations , 1983 .

[6]  C. Ian Connolly,et al.  Cumulative generation of octree models from range data , 1984, ICRA.

[7]  L. Guibas,et al.  Primitives for the manipulation of general subdivisions and the computation of Voronoi , 1985, TOGS.

[8]  P. Besl.  Surfaces in early range image understanding , 1986 .

[9]  M. Hebert,et al.  The Representation, Recognition, and Locating of 3-D Objects , 1986 .

[10]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[11]  Charles T. Loop,et al.  Smooth Subdivision Surfaces Based on Triangles , 1987 .

[12]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[13]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[14]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[15]  Howard Wainer,et al.  Robust Regression & Outlier Detection , 1988 .

[16]  J. K. Aggarwal,et al.  Generation of volume/surface octree from range data , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Paul J. Besl,et al.  Surfaces in Range Image Understanding , 1988, Springer Series in Perception Engineering.

[18]  Paul J. Besl,et al.  The Free-Form Surface Matching Problem , 1990 .

[19]  P. Rousseeuw,et al.  Unmasking Multivariate Outliers and Leverage Points , 1990 .

[20]  Yoshio Ohno,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[21]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[23]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[24]  John M. Snyder,et al.  Interval analysis for computer graphics , 1992, SIGGRAPH.

[25]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[26]  G. Champleboux,et al.  From accurate range imaging sensor calibration to accurate model-based 3D object localization , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[29]  Tony DeRose,et al.  Mesh optimization , 1993, SIGGRAPH.

[30]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[32]  J.-M. Vezien,et al.  Multiple representation approach to geometric model construction from range data , 1994, Proceedings of 1994 IEEE 2nd CAD-Based Vision Workshop.

[33]  Yang Chen.  Description of complex objects using multiple range images , 1994 .

[34]  Martin Rutishauser,et al.  Merging range images of arbitrarily shaped objects , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Tony DeRose,et al.  Piecewise smooth surface reconstruction , 1994, SIGGRAPH.

[36]  An Li,et al.  Octree encoding of objects from range images , 1994, Pattern Recognit..

[37]  M. Hebert,et al.  Merging multiple views using a spherical representation , 1994, Proceedings of 1994 IEEE 2nd CAD-Based Vision Workshop.

[38]  Allan D. Jepson,et al.  A new closed-form solution for absolute orientation , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Robert Bergevin,et al.  Registration of multiple range views for automatic 3-D model building , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Marc Rioux,et al.  Three-dimensional registration using range and intensity information , 1994, Other Conferences.

[41]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[42]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[43]  R. Bergevin,et al.  Registering range views of multipart objects , 1995 .

[44]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[45]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[46]  Martin D. Levine,et al.  Registering Multiview Range Data to Create 3D Computer Objects , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Marc Levoy,et al.  Better optical triangulation through spacetime analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[48]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[49]  Katsushi Ikeuchi,et al.  A Spherical Representation for Recognition of Free-Form Surfaces , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Takeo Kanade,et al.  A multibaseline stereo system with active illumination and real-time image acquisition , 1995, Proceedings of IEEE International Conference on Computer Vision.

[51]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Kari Pulli,et al.  Surface reconstruction from misregistered data , 1995, Optics & Photonics.

[53]  Nelson L. Max,et al.  Rendering Trees from Precomputed Z-Buffer Views , 1995, Rendering Techniques.

[54]  Glenn H. Tarbox,et al.  IVIS: An Integrated Volumetric Inspection System , 1995, Comput. Vis. Image Underst..

[55]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[56]  Naokazu Yokoya,et al.  A Robust Method for Registration and Segmentation of Multiple Range Images , 1995, Comput. Vis. Image Underst..

[57]  David Salesin,et al.  Hierarchical image caching for accelerated walkthroughs of complex environments , 1996, SIGGRAPH.

[58]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[59]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[60]  S. Sardy,et al.  Triogram Models , 1996 .

[61]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[62]  Richard Pito,et al.  Mesh integration based on co-measurements , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[63]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[64]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[65]  Reinhard Klette,et al.  Handbook of image processing operators , 1996 .

[66]  T. Evgeniou.  Image Based Rendering Using Algebraic Techniques , 1996 .

[67]  Adrian Hilton,et al.  Reliable Surface Reconstructiuon from Multiple Range Images , 1996, ECCV.

[68]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[69]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[70]  Andrew W. Fitzgibbon,et al.  Simultaneous registration of multiple range views for use in reverse engineering , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[71]  Gang Wang,et al.  From images to models: automatic 3D object model construction from multiple views , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[72]  Adrian Hilton,et al.  Registration of multiple point sets , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[73]  Katsuhiko Sakaue,et al.  Registration and integration of multiple range images for 3-D model construction , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[74]  Tien-Tsin Wong,et al.  Image-based Rendering with Controllable Illumination , 1997, Rendering Techniques.

[75]  Linda G. Shapiro,et al.  Robust meshes from multiple range maps , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[76]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[77]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[78]  Jochen Wingbermühle,et al.  Automatic reconstruction of 3D objects using a mobile monoscopic camera , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[79]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[80]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[81]  Hajime Terasaki,et al.  A portable three-dimensional digitizer , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[82]  Linda G. Shapiro,et al.  View-base Rendering: Visualizing Real Objects from Scanned Range and Color Data , 1997, Rendering Techniques.

[83]  B. Curless.  New Methods for Surface Reconstruction from Range Images , 1997 .

[84]  Andrew E. Johnson,et al.  Surface registration by matching oriented points , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[85]  Amitabh Varshney,et al.  Navigating static environments using image-space simplification and morphing , 1997, SI3D.

[86]  Linda G. Shapiro,et al.  Surface Modeling and Display from Range and Color Data , 1997, ICIAP.

[87]  Jovan Popovic,et al.  Progressive simplicial complexes , 1997, SIGGRAPH.

[88]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[89]  Sebastian Weik,et al.  Registration of 3-D partial surface models using luminance and depth information , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[90]  Lance Williams,et al.  View interpolation for image synthesis , 1998 .

[91]  Linda G. Shapiro,et al.  Acquisition and visualization of colored 3D objects , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[92]  William H. Press,et al.  Numerical recipes in C , 2002 .