Image-flow computation: An estimation-theoretic framework and a unified perspective

Abstract Image flow is a major source of three-dimensional information. This paper describes a new framework for computing image flow from time-varying imagery. In this framework, image-flow information is classified into two categories—conservation information and neighborhood information. Each type of information is recovered in the form of an estimate accompanied by a covariance matrix. Image flow is then computed by fusing the two estimates using estimation-theoretic techniques. This framework offers the following principal advantages. First, it allows estimation of certain types of discontinuous flow fields without any a priori knowledge about the location of discontinuities. The flow fields thus recovered are not blurred at motion boundaries. Second, covariance matrices (or alternatively, confidence measures) are associated with the estimate of image flow at each stage of computation. The estimation-theoretic nature of the framework and its ability to provide covariance matrices make it very useful in the context of applications such as incremental estimation of scene depth using techniques based on Kalman filtering. Finally, this framework serves to unify various existing approaches for image-flow estimation. In this paper, two algorithms based on this framework are described and are used to recover image flow from a variety of image sequences. To illustrate the usefulness of the framework in the context of an application, the image-flow estimates and their covariance matrices recovered from some representative image sequences are also used to recover scene depth.