Incremental Reconstruction of 3D Scenes from Multiple, Complex Images

Abstract The 3D Mosaic system is a vision system that incrementally reconstructs complex 3D scenes from a sequence of images obtained from multiple viewpoints. The system encompasses several levels of the vision process, starting with images and ending with symbolic scene descriptions. This paper describes the various components of the system, including stereo analysis, monocular analysis, and constructing and updating the scene model. In addition, the representation of the scene model is described. This model is intended for tasks such as matching, display generation, planning paths through the scene, and making other decisions about the scene environment. Examples showing how the system is used to interpret complex aerial photographs of urban scenes are presented. Each view of the scene, which may be either a single image or a stereo pair, undergoes analysis which results in a 3D wire-frame description that represents portions of edges and vertices of objects. The model is a surface-based description constructed from the wire frames. With each successive view, the model is incrementally updated and gradually becomes more accurate and complete. Task-specific knowledge, involving block-shaped objects in an urban scene, is used to extract the wire frames and construct and update the model. The model is represented as a graph in terms of symbolic primitives such as faces, edges, vertices, and their topology and geometry. This permits the representation of partially complete, planar-faced objects. Because incremental modifications to the model must be easy to perform, the model contains mechanisms to (1) add primitives in a manner such that constraints on geometry imposed by these additions are propagated throughout the model, and (2) modify and delete primitives if discrepancies arise between newly derived and current information. The model also contains mechanisms that permit the generation, addition, and deletion of hypotheses for parts of the scene for which there is little data.

[1]  Takeo Kanade,et al.  Incremental Acquisition of a Three-Dimensional Scene Model from Images , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  S. Suess,et al.  Three dimensional modelling , 1974 .

[3]  Jon Doyle.  A truth maintenance system , 1981 .

[4]  Jon Doyle,et al.  Three Short Essays on Decisions, Reasons, and Logics, , 1981 .

[5]  ARISTIDES A. G. REQUICHA,et al.  Representations for Rigid Solids: Theory, Methods, and Systems , 1980, CSUR.

[6]  David Alan Bourne,et al.  Fault Detection In Manufacturing Cells Based On Three-Dimensional Visual Information , 1982, Other Conferences.

[7]  Marsha Jo Hannah,et al.  Computer matching of areas in stereo images. , 1974 .

[8]  Thomas O. Binford,et al.  The Interpretation of Three-Dimensional Structure from Image Curves , 1981, IJCAI.

[9]  William B. Thompson,et al.  Disparity Analysis of Images , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  John R. Kender.  Environmental Labelings in Low-Level Image Understanding , 1983, IJCAI.

[12]  Steven M. Rubin,et al.  Natural scene recognition using locus search , 1980 .

[13]  Takeo Kanade,et al.  Recovery of the Three-Dimensional Shape of an Object from a Single View , 1981, Artif. Intell..

[14]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[15]  Michael Potmesil,et al.  Generating Models of Solid Objects by Matching 3D Surface Segments , 1983, IJCAI.

[16]  Bruce G. Baumgart,et al.  Geometric modeling for computer vision. , 1974 .

[17]  Thomas O. Binford,et al.  Depth from Edge and Intensity Based Stereo , 1981, IJCAI.

[18]  W E Grimson,et al.  A computer implementation of a theory of human stereo vision. , 1981, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[19]  S. Croucher,et al.  Surveys , 1965, Understanding Communication Research Methods.

[20]  Charles M. Eastman,et al.  Geometric modelling: a survey , 1979 .

[21]  Hans P. Moravec.  Obstacle avoidance and navigation in the real world by a seeing robot rover , 1980 .

[22]  Robert L. Henderson,et al.  Automatic Stereo Reconstruction Of Man-Made Targets , 1979, Other Conferences.

[23]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[24]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[25]  Robert C. Bolles,et al.  Experiments in Map-Guided Photo Interpretation , 1977, IJCAI.

[26]  Clarence L. Coates,et al.  Visual Learning from Multiple Views , 1975, IEEE Transactions on Computers.