Essential Matrix

When we examined the egomotion problem, we considered two neighboring frames of a video that were close in time so that the rotation and translation were very small. In the remaining lectures, we address a related problem, namely binocular stereo. Here, we do not assume that the translation and rotation between cameras is small, nor do we assume that the two cameras have the same internal parameters. Despite these differences, similarities between the problems remain. Recall that in the egomotion problem, the velocity vector at a pixel was the sum of a rotation component and a translation component, such that the translation component depended on inverse depth. The velocity vector at a point (x, y) was thus constrained to a line in (vx, vy) space, such that the direction of the line was in the direction of heading i.e. the direction of translation from one camera position to the next. (Recall Eqns. 1,2,4 from lecture 20.) As we see in this lecture, there is a similar constraint in binocular stereo – namely that constrains corresponding points to lie on a line.)