Image-based 3D photography using opacity hulls

We have built a system for acquiring and displaying high quality graphical models of objects that are impossible to scan with traditional scanners. Our system can acquire highly specular and fuzzy materials, such as fur and feathers. The hardware set-up consists of a turntable, two plasma displays, an array of cameras, and a rotating array of directional lights. We use multi-background matting techniques to acquire alpha mattes of the object from multiple viewpoints. The alpha mattes are used to construct an opacity hull. The opacity hull is a new shape representation, defined as the visual hull of the object with view-dependent opacity. It enables visualization of complex object silhouettes and seamless blending of objects into new environments. Our system also supports relighting of objects with arbitrary appearance using surface reflectance fields, a purely image-based appearance representation. Our system is the first to acquire and render surface reflectance fields under varying illumination from arbitrary viewpoints. We have built three generations of digitizers with increasing sophistication. In this paper, we present our results from digitizing hundreds of models.

[1]  Linda G. Shapiro,et al.  View-base Rendering: Visualizing Real Objects from Scanned Range and Color Data , 1997, Rendering Techniques.

[2]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[3]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[4]  Wei-Chao Chen,et al.  Light field mapping: efficient representation and hardware rendering of surface light fields , 2002, SIGGRAPH.

[5]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[6]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[7]  Gabriel Taubin,et al.  Acquiring Input for Rendering at Appropriate Levels of Detail: Digitizing a Tietà , 1998, Rendering Techniques.

[8]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[9]  David J. Kriegman,et al.  Illumination-based image synthesis: creating novel images of human faces under differing pose and lighting , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[10]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[11]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[12]  Katsushi Ikeuchi,et al.  Appearance compression and synthesis based on 3D model for mixed reality , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[14]  Katsushi Ikeuchi,et al.  Eigen-texture method: Appearance compression based on 3D model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[16]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[17]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.

[18]  Paul E. Debevec,et al.  A photometric approach to digitizing cultural artifacts , 2001, VAST '01.

[19]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[20]  Matthias Zwicker,et al.  Surfels: surface elements as rendering primitives , 2000, SIGGRAPH.

[21]  Pedro V. Sander,et al.  Silhouette clipping , 2000, SIGGRAPH.

[22]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[23]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[24]  Adam Finkelstein,et al.  Real-time fur over arbitrary surfaces , 2001, I3D '01.

[25]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[26]  Jayaram K. Udupa,et al.  Shell rendering , 1993, IEEE Computer Graphics and Applications.

[27]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[28]  Matthias Zwicker,et al.  Surface splatting , 2001, SIGGRAPH.

[29]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[30]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Gavin S. P. Miller,et al.  Lazy Decompression of Surface Light Fields for Precomputed Global Illumination , 1998, Rendering Techniques.

[32]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[33]  Hans-Peter Seidel,et al.  Image-Based Reconstruction of Spatially Varying Materials , 2001 .

[34]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[35]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[36]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[37]  Marc Levoy,et al.  The Use of Points as a Display Primitive , 2000 .

[38]  David J. Kriegman,et al.  Image-based modeling and rendering of surfaces with arbitrary BRDFs , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[40]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.