Automatic Reconstruction of Buildings from Stereoscopic Image Sequences

A vision–based 3‐D scene analysis system is described that is capable to model complex real–world scenes like streets and buildings automatically from stereoscopic image pairs. Input to the system is a sequence of stereoscopic images taken with two standard CCD Cameras and TV lenses. The relative orientation of both cameras to each other is known by calibration. The camerapair is then moved throughout the scene and a long sequence of closely spaced views is recorded. Each of the stereoscopic image pairs is rectified and a dense map of 3‐D suface points is obtained by area correlation, object segmentation, interpolation, and triangulation. 3‐D camera motion relative to the scene coordinate system is tracked directly from the image sequence which allows to fuse 3‐D surface measurements from different viewpoints into a consistent 3‐D model scene. The surface geometry of each scene object is approximated by a triangular surface mesh which stores the suface texture in a texture map. From the textured 3‐D models, realistic looking image sequences from arbitrary view points can be synthesized using computer graphics.