Tutorial: Computer vision—towards a three-dimensional world

Abstract This paper surveys theory and techniques relevant to the design of three-dimensional computer vision systems for intelligent robots and other applications. Being different from two-dimensional vision, 3-D vision uses information not only about the projected boundaries of objects, but also about the shapes of their surfaces, and the ranges between the objects and the camera. The range information may be acquired either by means of direct measurements based on laser light or ultrasound reflectance, or by indirect computational approaches, such as stereo vision or structural lighting. The calibration of camera systems and space coordinates is necessary if vision is to be used for the precise location of specific targets. Information about surface shape may be recovered from the variation of brightness in images, and methods used for such purposes are often called ‘shape from shading’. Shape can also be recovered by detecting the deformation of contours or textures on the object's surfaces, or by calculating the optical flow diagrams for moving objects. From time-varying image sequences, motion parameters (including rotation and translation in 3-D space) can be estimated, based on assumptions of rigidity of objects. Two approaches, feature-based and optical-flow-based, may be used for this task. Turning to practical applications, it is important that 3-D vision systems should be able to recognize automatically the objects viewed in a scene. For recognition, 3-D objects should first be modelled in terms of appropriate parameters or data structures, in order that matching procedures may be performed between unknown objects and modelled objects. Appropriate techniques, and the principles on which they are based, are reviewed. As an example, a stereo computer vision system for recognition and location of polyhedral objects is illustrated.