Linear and bilinear subspace methods for structure from motion

Structure from Motion (SFM), which is recovering camera motion and scene structure from image sequences, has various applications, such as scene modeling, robot navigation and object recognition. Most of previous research on SFM requires simplifying assumptions on the camera or the scene. Common assumptions are (a) the camera intrinsic parameters, such as focal lengths, are known or unchanged throughout the sequence, and/or (b) the scene does not contain moving objects. In practice, these are unrealistic assumptions. In this thesis we present a collection of reconstruction methods for dealing with image sequences taken with uncalibrated cameras and/or of multiple motion scenes. The methods produce Euclidean reconstruction directly from feature point locations and are based on the bilinear relationship of camera motion and scene structure. For uncalibrated image sequences, we embed the camera intrinsic parameters within the camera motion representation. For image sequences of multiple motion scenes, we incorporate multiple motions into the scene structure representation. In this way, we derive linear and bilinear subspace constraints on the large amount of information integrated over the entire image sequences. By taking advantage of this redundant information we can achieve accurate and reliable reconstruction. Firstly, we propose a uncalibrated Euclidean reconstruction method from multiple uncalibrated views. This method first performs a projective reconstruction using a bilinear factorization algorithm, and then converts the projective solution to a Euclidean one by enforcing metric constraints. We present three normalization algorithms to generate the Euclidean reconstruction and the intrinsic parameters. The first two algorithms are linear, one for dealing with the case that only the focal lengths are unknown, and another for the case that the focal lengths and the constant principal point are unknown. The third algorithm is bilinear, dealing with the case that the focal lengths, the principal points and the aspect ratios are all unknown. Secondly, we present a linear method to reconstruct a scene containing multiple moving objects together with the camera motion. The number of the moving objects is automatically detected without prior motion segmentation. Assuming that the objects are moving linearly with constant speeds, we propose a unified geometrical representation of the static scene and the moving objects. This representation enables the embedding of the linear motion constraints into the scene structure, which naturally leads to a factorization-based method. Thirdly, we describe a method for multiple motion scene reconstruction from uncalibrated views. The method recovers the scene structure, the trajectories of the moving objects and the camera intrinsic (except skews) and extrinsic parameters simultaneously assuming that the objects are moving with constant velocities. We embed the assumptions within the scene representation and therefore propose a bilinear factorization algorithm to generate a projective reconstruction, and then impose metric constraints to compute the Euclidean reconstruction and the camera intrinsic parameters. We also discuss other issues related to the accuracy and reliability of these reconstruction methods, such as minimum data requirement and gauge selection. The reconstruction methods have been tested on a series of synthetic sequences to evaluate the quality of the methods, and real image sequences to demonstrate their applicability.