Dynamic reconstruction and integration of 3D structure information

This thesis presents a computational scheme for dynamic reconstruction of 3D structure and 3D motion from instantaneous motion in the 2D images. The scheme consists of multiple computational stages and their interactions. For these computational stages, we develop the following algorithms. First, structure-from-motion algorithms are developed that recover 3D structure and 3D velocities from a single image velocity field by optimizing constraints from image motion and the rigidity constraint. The algorithms allow variable weighting of the strength of rigidity between pairs of points so that they can cope with nonrigid objects and multiple rigid objects. Second, a temporal integration algorithm is developed that combines information from multiple frames for robust and reliable recovery of 3D structure. The algorithm is based on the Kalman filter and sequentially predicts and updates the current estimates of 3D structure as new information is obtained. Third, surface reconstruction algorithms are developed that simultaneously group and approximate sparse and noisy data points for reconstructing multiple surfaces, such as transparent or occluded surfaces. The surface reconstruction algorithms are combined with the structure-from-motion algorithms and the temporal integration algorithm for more robust recovery of 3D structure from motion. 3D structure information and occluding boundary information from various sources can also be integrated to form a coherent surface structure. Through computer simulations, we show that the following properties of the proposed computational scheme are consistent with human perception of 3D structure from motion. First, relative depths of an object are efficiently recovered from instantaneous image velocities using the rigidity constraint. Second, the quality of recovered 3D structure incrementally improves over an extended time as more frames are obtained. Third, incorporation of a surface reconstruction process further improves the quality of recovered 3D structure over an extended time, particularly when an individual feature point has only a short lifetime. Multiple transparent surfaces are recovered by classifying the data points into multiple groups. The use of an occluding boundary constraint can also influence the surface reconstruction. Finally, in the case of multiple rigid objects and nonrigid objects, relaxing the rigidity constraint and limiting the spatial extent of rigid interaction yield interesting 3D structures of these objects that are consistent with what human observers perceive. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)