Three-Dimensional Scene Reconstruction: A Review of Approaches

The production of 3D models has been a popular research topic already for a long time, and important progress has been made since the early days. During the last decades, vision systems have established to become the standard and one of the most efficient sensorial assets in industrial and everyday applications. Due to the fact that vision provides several vital attributes, many applications tend to use novel vision systems into domestic, working, industrial, and any other environments. To achieve such goals, a vision system should robustly and effectively reconstruct the 3D surface and the working space. This chapter discusses different methods for capturing the three-dimensional surface of a scene. Geometric approaches to three-dimensional scene reconstruction are generally based on the knowledge of the scene structure from the camera’s internal and external parameters. Another class of methods encompasses the photometric approaches, which evaluate the pixels’ intensity to understand the three-dimensional scene structure. The third and final category of approaches, the so-called real aperture approaches, includes methods that use the physical properties of the visual sensors for image acquisition in order to reproduce the depth information of a scene. DOI: 10.4018/978-1-61350-326-3.ch008

[1]  Trevor Darrell,et al.  Inferring 3D structure with a statistical image-based shape model , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[2]  A. N. Rajagopalan,et al.  Real Aperture Axial Stereo: Solving for Correspondences in Blur , 2009, DAGM-Symposium.

[3]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[4]  Lu Jiang,et al.  3D Surface Reconstruction and Analysis in Automated Apple Stem-End/Calyx Identification , 2009 .

[5]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Stefano Soatto,et al.  Autocalibration and Uncalibrated Reconstruction of Shape from Defocus , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  David A. Forsyth,et al.  Shape from Texture without Boundaries , 2002, International Journal of Computer Vision.

[8]  Shree K. Nayar,et al.  Real-Time Focus Range Sensor , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Richard Szeliski,et al.  Fast shape from shading , 1990, CVGIP Image Underst..

[11]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[12]  Sagarmay Deb Multimedia Systems and Content-Based Image Retrieval , 2003 .

[13]  J. Todd,et al.  The effects of viewing angle, camera angle, and sign of surface curvature on the perception of three-dimensional shape from texture. , 2007, Journal of vision.

[14]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Jörg Krüger,et al.  Shape from Shading Using Probability Functions and Belief Propagation , 2009, International Journal of Computer Vision.

[16]  Nir A. Sochen,et al.  Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Jean-Philippe Pons,et al.  Spatio-Temporal Shape from Silhouette using Four-Dimensional Delaunay Meshing , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Antonios Gasteratos,et al.  Review of Stereo Vision Algorithms: From Software to Hardware , 2008 .

[20]  João Manuel R. S. Tavares,et al.  Three-dimensional reconstruction and characterization of human external shapes from two-dimensional images using volumetric methods , 2010, Computer methods in biomechanics and biomedical engineering.

[21]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[22]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Ira Kemelmacher-Shlizerman,et al.  Photometric Stereo with General, Unknown Lighting , 2006, International Journal of Computer Vision.

[24]  Donald B. Gennery,et al.  Generalized Camera Calibration Including Fish-Eye Lenses , 2006, International Journal of Computer Vision.

[25]  Nassir Navab,et al.  Efficient visual hull computation for real-time 3D reconstruction using CUDA , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[26]  Ryan White,et al.  Capturing and animating occluded cloth , 2007, SIGGRAPH 2007.

[27]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[28]  Yanxi Liu,et al.  A computational model for periodic pattern perception based on frieze and wallpaper groups , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Sang-Heon Lee,et al.  Fusion on Citrus Image Data from Cold Mirror Acquisition System , 2012, Int. J. Comput. Vis. Image Process..

[30]  Eduardo Zalama Casanova,et al.  Obtaining 3D models of indoor environments with a mobile robot by estimating local surface directions , 2004, Robotics Auton. Syst..

[31]  W. James MacLean,et al.  Leveraging cost matrix structure for hardware implementation of stereo disparity computation using dynamic programming , 2010, Comput. Vis. Image Underst..

[32]  Alexei A. Efros,et al.  Discovering Texture Regularity as a Higher-Order Correspondence Problem , 2006, ECCV.

[33]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Paolo Favaro,et al.  Shape from Focus and Defocus: Convexity, Quasiconvexity and Defocus-Invariant Textures , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[35]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[36]  Chi Zhang,et al.  Real-time 3D shape inspection system of automotive parts based on structured light pattern , 2011 .

[37]  Tae-Sun Choi,et al.  A novel iterative shape from focus algorithm based on combinatorial optimization , 2010, Pattern Recognit..

[38]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[39]  Kun Zhou,et al.  Data-Parallel Octrees for Surface Reconstruction , 2011, IEEE Transactions on Visualization and Computer Graphics.

[40]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[41]  A. N. Rajagopalan,et al.  Improving Shape From Focus Using Defocus Cue , 2007, IEEE Transactions on Image Processing.

[42]  Daniel Cremers,et al.  Integration of Multiview Stereo and Silhouettes Via Convex Functionals on Convex Domains , 2008, ECCV.

[43]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[44]  Jose Garcia-Rodriguez,et al.  Robotic Vision: Technologies for Machine Learning and Vision Applications , 2013 .

[45]  Pietro Perona,et al.  Evaluation of Features Detectors and Descriptors based on 3D Objects , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[46]  Aboul Ella Hassanien,et al.  A Feature Selection Approach for Network Intrusion Classification: The Bi-Layer Behavioral-Based , 2013, Int. J. Comput. Vis. Image Process..

[47]  W. James MacLean,et al.  A Real-Time Large Disparity Range Stereo-System Using FPGAs , 2006, ACCV.

[48]  Berthold K. P. Horn Height and gradient from shading , 1989, International Journal of Computer Vision.

[49]  Avinash C. Kak,et al.  Vision for Mobile Robot Navigation: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Laurent Jacques,et al.  Shape from texture for omnidirectional images , 2008, 2008 16th European Signal Processing Conference.

[51]  Daniel G. Aliaga,et al.  An Adaptive Correspondence Algorithm for Modeling Scenes with Strong Interreflections , 2009, IEEE Transactions on Visualization and Computer Graphics.

[52]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[53]  Stephen T. C. Wong,et al.  Robust 3D reconstruction and identification of dendritic spines from optical microscopy imaging , 2009, Medical Image Anal..

[54]  A. N. Rajagopalan,et al.  Discontinuity-Adaptive Shape from Focus Using a Non-convex Prior , 2009, DAGM-Symposium.

[55]  Juho Kannala,et al.  A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Yanchun Zhang,et al.  Algorithm for the Retrieval of Image with Emergence Index in Multimedia , 2004 .

[57]  Yücel Yemez,et al.  A volumetric fusion technique for surface reconstruction from silhouettes and range data , 2007, Comput. Vis. Image Underst..

[58]  Jing Xu,et al.  Windshield shape inspection using structured light patterns from two diffuse planar light sources , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[59]  Stefano Soatto,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE IEEE TRANSACTION OF PATTERN RECO , 2022 .

[60]  Xin Liu,et al.  Shape from silhouettes based on a centripetal pentahedron model , 2008, Graph. Model..

[61]  Antonios Gasteratos,et al.  Obtaining Reliable Depth Maps for Robotic Applications from a Quad-Camera System , 2009, ICIRA.

[62]  Simon Lacroix,et al.  Vision-Based SLAM: Stereo and Monocular Approaches , 2007, International Journal of Computer Vision.

[63]  Takeo Kanade,et al.  Shape from interreflections , 2004, International Journal of Computer Vision.

[64]  S. Grossberg,et al.  A neural model of 3D shape-from-texture: Multiple-scale filtering, boundary grouping, and surface filling-in , 2007, Vision Research.

[65]  Robert J. Woodham,et al.  Analysing Images of Curved Surfaces , 1981, Artif. Intell..

[66]  Upendra Kumar,et al.  Significant Enhancement of Object Recognition Efficiency Using Human Cognition based Decision Clustering , 2013, Int. J. Comput. Vis. Image Process..

[67]  Dejan Seatovic A Segmentation Approach in Novel Real Time 3D Plant Recognition System , 2008, ICVS.

[68]  Angeline M. Loh,et al.  Shape from Non-homogeneous, Non-stationary, Anisotropic, Perspective Texture , 2005, BMVC.

[69]  Yanxi Liu,et al.  Near-regular texture analysis and manipulation , 2004, SIGGRAPH 2004.

[70]  Peter F. Sturm,et al.  A Generic Concept for Camera Calibration , 2004, ECCV.

[71]  Joan Lasenby,et al.  Shape from Texture of Developable Surfaces Via Fourier Analysis , 2007, ISVC.

[72]  Georgios Ch. Sirakoulis,et al.  Intelligent robots need intelligent vision : Visual 3D perception , 2008 .

[73]  Shiu Yin Yuen,et al.  Recovering Shape by Shading and Stereo Under Lambertian Shading Model , 2009, International Journal of Computer Vision.

[74]  Philippe Martinet,et al.  A generic fisheye camera model for robotic applications , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Matthew Harker,et al.  Least squares surface reconstruction from measured gradient fields , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[78]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[79]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[80]  Peter F. Sturm,et al.  General Imaging Geometry for Central Catadioptric Cameras , 2008, ECCV.

[81]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[82]  Fridtjof Stein,et al.  In-factory calibration of multiocular camera systems , 2004, SPIE Photonics Europe.

[83]  O. Tepper,et al.  Three‐Dimensional Imaging Provides Valuable Clinical Data to Aid in Unilateral Tissue Expander‐Implant Breast Reconstruction , 2008, The breast journal.

[84]  Shana Smith,et al.  Shape Similarity Matching With Octree Representations , 2009, J. Comput. Inf. Sci. Eng..

[85]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[86]  Zhifeng Liu,et al.  Dynamic Programming Stereo on Real-World Sequences , 2008, ICONIP.

[87]  C. Angulo,et al.  Uniform sampling of rotations for discrete and continuous learning of 2D shape models , 2012 .

[88]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[89]  Jing Li,et al.  A comprehensive review of current local features for computer vision , 2008, Neurocomputing.

[90]  Antonios Gasteratos,et al.  Stereovision-Based Algorithm for Obstacle Avoidance , 2009, ICIRA.