Interactive modeling from dense color and sparse depth

We are developing a system for interactive modeling of real world scenes. The acquisition device consists of a video camera enhanced with an attached laser system. As the operator sweeps the scene, the device acquires dense color and sparse depth frames that are registered and merged into a point-based model. The evolving model is rendered continually to provide immediate operator feedback. The interactive modeling pipeline runs at five frames per second. We model scenes in two modes based on their geometric complexity. Scenes that contain large smooth surfaces are modeled freehand; scenes that contain small uneven surfaces are modeled using a parallax-free camera bracket.

[1]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[2]  Ruzena Bajcsy,et al.  Occlusions as a Guide for Planning the Next View , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[4]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[5]  Giancarlo Ferrigno,et al.  Autoscan: A Flexible and Portable 3D Scanner , 1998, IEEE Computer Graphics and Applications.

[6]  Gabriel Taubin,et al.  Building a Digital Model of Michelangelo's Florentine Pietà , 2002, IEEE Computer Graphics and Applications.

[7]  Voicu Popescu,et al.  The ModelCamera: a hand-held device for interactive modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[8]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[9]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[10]  Luc Van Gool,et al.  A guided tour to virtual Sagalassos , 2001, VAST '01.

[11]  Anselmo Lastra,et al.  Monticello Through the Window , 2003, VAST.

[12]  W.R. Scott,et al.  View planning with a registration constraint , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[13]  Roger J. Hubbold,et al.  Hybrid Geometric — Image Based Rendering , 2002, Comput. Graph. Forum.

[14]  Voicu Popescu,et al.  Interactive Point-Based Modeling from Dense Color and Sparse Depth , 2004, PBG.

[15]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[16]  Andrew Jones,et al.  Digital Reunification of the Parthenon and its Sculptures , 2003, VAST.

[17]  Luc Van Gool,et al.  From images to 3D models , 2002, CACM.

[18]  Patrick Hébert.  A self-referenced hand-held range sensor , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[19]  Hisham El-Shishiny,et al.  Scanning and processing 3D objects for Web display , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[20]  Svetha Venkatesh,et al.  Low-cost interactive active monocular range finder , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[21]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[22]  Ken-ichi Anjyo.  Tour into the picture , 1997, SIGGRAPH '97.

[23]  Voicu Popescu,et al.  The WarpEngine: an architecture for the post-polygonal age , 2000, SIGGRAPH.

[24]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[25]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[26]  Matthias Zwicker,et al.  Surfels: surface elements as rendering primitives , 2000, SIGGRAPH.

[27]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[28]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[29]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[30]  Pietro Perona,et al.  3D Photography Using Shadows in Dual-Space Geometry , 1999, International Journal of Computer Vision.

[31]  Voicu Popescu,et al.  High Quality 3D Image Warping by Separating Visibility from Reconstruction , 1999 .

[32]  Giancarlo Ferrigno,et al.  AUTOSCAN: A flexible and portable scanner of 3D surfaces , 1998 .

[33]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.

[34]  Luc Van Gool,et al.  Real-time range scanning of deformable surfaces by adaptively coded structured light , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[35]  Ioannis Stamos,et al.  View Planning for Site Modeling , 2000 .

[36]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..