Gramians, generalized inverses, and the least-squares approximation of optical flow

Abstract This paper deals with the recovery of optical flow, that is to say, with the identification of a vector field, defined on some subset of the image plane, which accounts for the infinitesimal time evolution of the image of a particular object. Our formulation is general in that it allows for the vector field to be expressed as a linear combination of a fixed set of vector fields and it allows the measurements to include (a) the velocity of feature points, (b) the velocity normal to an evolving contour, and/or (c) the velocity tangent to an intensity gradient. The method is based on least squares and an explicit formula for the generalized inverse of a class of integral operators. It involves a gramian whose invertibility is necessary and sufficient for the identification of a unique best-fitting vector field. Various important subcases have been studied earlier and reported in the computer vision literature; the emphasis here is on the systematic development of a general tool.