3D Content Creation by Passive Optical Methods

The possibility of obtaining 3D models, i.e., mathematical descriptions of real objects or scenes has paved the way to a wide range of new and exciting applications in fields such as virtual simulation, human-computer interaction (HCI), scientific visualization, cultural heritage documentation, medicine, industrial prototyping, reverse engineering, entertainment (movies and video games), web-based commerce and marketing, just to name a few. The construction of the 3D model of a real object or scene by optical sensors, also referred to as 3D modeling pipeline, essentially consists of four steps: 1) data acquisition, 2) calibration, 3) reconstruction, and 4) model editing. Any optical sensing device used to collect data can only capture the surface front side and not what is occluded by it. Therefore, a full model must be built from a number of images covering the entire object (data acquisition). In order to perform 3D reconstruction, the camera’s parameters must be estimated by a procedure called calibration. Such information can also be obtained from the acquired images if they represent some common regions (by a procedure which is typically called self-calibration). Reconstruction is then performed and the resulting model is stored in an efficient description such as polygonal meshes, implicit surfaces, depth maps or volumetric descriptions. In practical situations, reconstruction may lead to models with some imperfections; thus, a further repairing step is recommend (model editing) [Davis et al. (2002); Levoy et al. (2000)].

[1]  Laurent D. Cohen,et al.  On active contour models and balloons , 1991, CVGIP Image Underst..

[2]  Thomas O. Binford,et al.  Local shape from specularity , 1988, Comput. Vis. Graph. Image Process..

[3]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[4]  Laurent Vinet,et al.  Hierarchical region based stereo matching , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  J. Koenderink,et al.  Irradiation direction from texture. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[6]  Christian Wöhler 3D surface reconstruction by self-consistent fusion of shading and shadow features , 2004, ICPR 2004.

[7]  Nahum Kiryati,et al.  Photometric stereo under perspective projection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  V. Caselles,et al.  A geometric model for active contours in image processing , 1993 .

[9]  Frederic Fol Leymarie,et al.  Tracking Deformable Objects in the Plane Using an Active Contour Model , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[11]  Maarten Vergauwen,et al.  A Hierarchical Symmetric Stereo Algorithm Using Dynamic Programming , 2002, International Journal of Computer Vision.

[12]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[14]  Demetri Terzopoulos,et al.  Deformable models , 2000, The Visual Computer.

[15]  Luca Ballan,et al.  Multimodal 3D Shape Recovery from Texture, Silhouette and Shadow Information , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[16]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[17]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ruigang Yang,et al.  Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Michael Potmesil Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[20]  J. Kender,et al.  Shape from darkness under error , 1996 .

[21]  Roberto Cipolla,et al.  The visual motion of curves and surfaces , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[22]  William Ribarsky,et al.  Toward Spontaneous Interaction with the Perceptive Workbench , 2000, IEEE Computer Graphics and Applications.

[23]  Holly E. Rushmeier,et al.  The 3D Model Acquisition Pipeline , 2002, Comput. Graph. Forum.

[24]  Roberto Cipolla,et al.  Reconstruction in the Round Using Photometric Normals and Silhouettes. , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  A D Marshall,et al.  Geometric Modelling for Computer Vision , 1992 .

[26]  Charles Fox,et al.  An introduction to the calculus of variations , 1950 .

[27]  Subhasis Chaudhuri,et al.  Depth From Defocus: A Real Aperture Imaging Approach , 1999, Springer New York.

[28]  Michael Heizmann,et al.  Fusion of Combined Stereo and Focus Series for Depth Estimation , 2006, GI Jahrestagung.

[29]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[30]  Marc Rioux,et al.  Beyond range sensing: XYZ-RGB digitizing and modeling , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[31]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[32]  Pietro Perona,et al.  Shadow Carving , 2001, ICCV.

[33]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[35]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[36]  Gregory Dudek,et al.  Out of the Dark: Using Shadows to Reconstruct 3D Surfaces , 1998, ACCV.

[37]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[38]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Hailin Jin,et al.  A Variational Approach to Shape from Defocus , 2002, ECCV.

[40]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[41]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[42]  Shree K. Nayar,et al.  Shape from Focus , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[44]  Stefano Soatto,et al.  3D shape from anisotropic diffusion , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[45]  Margaret M. Fleck A topological stereo matcher , 1991, International Journal of Computer Vision.

[46]  F. E. Nicodemus Reflectance nomenclature and directional reflectance and emissivity. , 1970, Applied optics.

[47]  John R. Kender,et al.  Shape from Darkness: Deriving Surface Information from Dynamic Shadows , 1986, AAAI.

[48]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[49]  Richard Szeliski,et al.  Tracking with Kalman snakes , 1993 .

[50]  Pietro Perona,et al.  3D Reconstruction by Shadow Carving: Theory and Practical Evaluation , 2007, International Journal of Computer Vision.

[51]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[52]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[53]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[54]  Kazufumi Kaneda,et al.  Dynamic contour: A texture approach and contour operations , 1995, The Visual Computer.

[55]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[56]  David A. Forsyth,et al.  Combining Cues: Shape from Shading and Texture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[57]  Demetri Terzopoulos,et al.  A dynamic finite element surface model for segmentation and tracking in multidimensional medical images with application to cardiac 4D image analysis. , 1995, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[58]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[59]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[60]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[61]  Takeo Kanade,et al.  Using shadows in finding surface orientations , 1983, Comput. Vis. Graph. Image Process..

[62]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[63]  Alex Pentland,et al.  3D structure from 2D motion , 1999, IEEE Signal Process. Mag..

[64]  Stefano Soatto,et al.  Multi-view stereo beyond Lambert , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[65]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[66]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[67]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[68]  Berthold K. P. Horn SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[69]  Hans-Peter Seidel,et al.  Hardware-Accelerated Visual Hull Reconstruction and Rendering , 2003, Graphics Interface.

[70]  John R. Kender,et al.  An optimal algorithm for the derivation of shape from shadows , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[71]  Jerry L. Prince,et al.  Snakes, shapes, and gradient vector flow , 1998, IEEE Trans. Image Process..

[72]  Bui Tuong Phong Illuminat~on for computer generated images , 1973 .

[73]  G. F. McLean,et al.  Line-Based Correction of Radial Lens Distortion , 1997, CVGIP Graph. Model. Image Process..

[74]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[75]  Roberto Cipolla,et al.  Using frontier points to recover shape, reflectance and illumination , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[76]  David J. Kriegman,et al.  What shadows reveal about object structure. , 2001 .

[77]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[78]  Paulo R. S. Mendonça,et al.  A simple technique for self-calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[79]  Emanuele Trucco,et al.  Rectification with unconstrained stereo geometry , 1997, BMVC.

[80]  David A. Forsyth Shape from Texture without Boundaries , 2002, ECCV.

[81]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[82]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[83]  Adrian Hilton,et al.  Multiple view reconstruction of people , 2004 .

[84]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.