Image-based reconstruction of spatial appearance and geometric detail

Real-world objects are usually composed of a number of different materials that often show subtle changes even within a single material. Photorealistic rendering of such objects requires accurate measurements of the reflection properties of each material, as well as the spatially varying effects. We present an image-based measuring method that robustly detects the different materials of real objects and fits an average bidirectional reflectance distribution function (BRDF) to each of them. In order to model local changes as well, we project the measured data for each surface point into a basis formed by the recovered BRDFs leading to a truly spatially varying BRDF representation. Real-world objects often also have fine geometric detail that is not represented in an acquired mesh. To increase the detail, we derive normal maps even for non-Lambertian surfaces using our measured BRDFs. A high quality model of a real object can be generated with relatively little input data. The generated model allows for rendering under arbitrary viewing and lighting conditions and realistically reproduces the appearance of the original object.

[1]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[2]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[3]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Hans-Peter Seidel,et al.  Image-Based BRDF Reconstruction , 1999 .

[5]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[6]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[7]  Toby Howard,et al.  Flexible Image‐Based Photometric Reconstruction using Virtual Light Sources , 2001, Comput. Graph. Forum.

[8]  Takeo Kanade,et al.  Shape from interreflections , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[9]  Hans-Peter Seidel,et al.  Towards interactive bump mapping with anisotropic shift-variant BRDFs , 2000, Workshop on Graphics Hardware.

[10]  Donald P. Greenberg,et al.  Non-linear approximation of reflectance functions , 1997, SIGGRAPH.

[11]  Katsushi Ikeuchi,et al.  Appearance Based Object Modeling using Texture Database: Acquisition Compression and Rendering , 2002, Rendering Techniques.

[12]  William H. Press,et al.  Numerical Recipes in Fortran 77: The Art of Scientific Computing 2nd Editionn - Volume 1 of Fortran Numerical Recipes , 1992 .

[13]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.

[15]  Wojciech Matusik,et al.  Acquisition and Rendering of Transparent and Refractive Objects , 2002, Rendering Techniques.

[16]  SeidelHans-Peter,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003 .

[17]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[18]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[19]  Gavin S. P. Miller,et al.  Lazy Decompression of Surface Light Fields for Precomputed Global Illumination , 1998, Rendering Techniques.

[20]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[21]  Hans-Peter Seidel,et al.  Image-Based Reconstruction of Spatially Varying Materials , 2001 .

[22]  Hans-Peter Seidel,et al.  Automated texture registration and stitching for real world models , 2000, Proceedings the Eighth Pacific Conference on Computer Graphics and Applications.

[23]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[24]  Stephen H. Westin,et al.  Predicting reflectance functions from complex surfaces , 1992, SIGGRAPH.

[25]  S. Marschner,et al.  Inverse Rendering for Computer Graphics , 1998 .

[26]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[27]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[28]  Eugene Fiume.  Alain Fournier, 1943–2000 An Appreciation , 2001 .

[29]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[30]  Hans-Peter Seidel,et al.  Entropy-Based Dark Frame Subtraction , 2001, PICS.

[31]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[33]  Andrea J. van Doorn,et al.  Bidirectional Reflection Distribution Function Expressed in Terms of Surface Scattering Modes , 1996, ECCV.

[34]  Gabriel Taubin,et al.  Acquiring Input for Rendering at Appropriate Levels of Detail: Digitizing a Tietà , 1998, Rendering Techniques.

[35]  J. Koenderink,et al.  Optical properties (bidirectional reflection distribution functions) of velvet. , 1998, Applied optics.

[36]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[37]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[38]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[39]  Paul A. Beardsley,et al.  Image-based 3D photography using opacity hulls , 2002, ACM Trans. Graph..

[40]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[41]  Harry Shum,et al.  Synthesizing bidirectional texture functions for real-world surfaces , 2001, SIGGRAPH.

[42]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[43]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[44]  Hans-Peter Seidel,et al.  Building a Photo Studio for Measurement Purposes , 2000, VMV.

[45]  Jitendra Malik,et al.  Recovering photometric properties of architectural scenes from photographs , 1998, SIGGRAPH.

[46]  Anselmo Lastra,et al.  Efficient rendering of spatial bi-directional reflectance distribution functions , 2002, HWWS '02.

[47]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[48]  Gabriel Taubin,et al.  Appying Shape from Lighting Variation to Bump Map Capture , 1997, Rendering Techniques.

[49]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[50]  Hans-Peter Seidel,et al.  A Silhouette-Based Algorithm for Texture Registration and Stitching , 2001, Graph. Model..

[51]  Robert L. Stevenson,et al.  Dynamic range improvement through multiple exposures , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[52]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[53]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[54]  Robert R. Lewis,et al.  Making Shaders More Physically Plausible , 1994, Comput. Graph. Forum.