Reconstruction from Two Calibrated Views

This chapter introduces the basic geometry of reconstruction of points in 3-D space from image measurements made from two different (calibrated) camera viewpoints. We then introduce a simple algorithm to recover the 3-D position of such points from their 2-D views. Although the algorithm is primarily conceptual, it illustrates the basic principles that will be used in deriving both more sophisticated algorithms as well as algorithms for the more general case of reconstruction from multiple views which will be discussed in Part III of the book. The framework for this chapter is rather simple: Assume that we are given two views of a set of N feature points, which we denote formally as I. The unknowns are the position of the points in 3-D space, denoted Z and the relative pose of the cameras, denoted G, we will in this Chapter first derive constraints between these quantities. These constraints take the general form (one for each feature point) for some functions f i. The central problem to be addressed in this Chapter is to start from this system of nonlinear equations and attempt, if possible, to solve it to recover the unknowns G and Z. We will show that it is indeed possible to recover the unknowns. The equations in (5.1) are nonlinear and no closed-form solution is known at this time. However, the functions f i have a very particular structure that allows us to eliminate the parameters Z that contain information about the 3-D structure of the scene and be left with a constraint on G and I alone, which is known as the epipolar constraint. As we shall see in Section 5.1, the epipolar constraint has the general form Interestingly enough, it can be solved for G in closed form, as we shall show in Section 5.2. The closed-form algorithm, however, is not suitable for use in real images corrupted by noise. In Section 5.3, we discuss how to modify it so as to minimize the effect of noise. When the two views are taken from infinitesimally close vantage points, the basic geometry changes in ways that we describe in Section 5.4.