A computational framework and an algorithm for the measurement of visual motion

The robust measurement of visual motion from digitized image sequences has been an important but difficult problem in computer vision. This paper describes a hierarchical computational framework for the determination of dense displacement fields from a pair of images, and an algorithm consistent with that framework. Our framework is based on a scale-based separation of the image intensity information and the process of measuring motion. The large-scale intensity information is first used to obtain rough estimates of image motion, which are then refined by using intensity information at smaller scales. The estimates are in the form of displacement (or velocity) vectors for pixels and are accompanied by a direction-dependent confidence measure. A smoothness constraint is employed to propagate measurements with high confidence to neighboring areas where the confidences are low. At all levels, the computations are pixel-parallel, uniform across the image, and based on information from a small neighborhood of a pixel. Results of applying our algorithm to pairs of real images are included. In addition to our own matching algorithm, we also show that two different hierarchical gradient-based algorithms are consistent with our framework.

[1]  H. Piaggio Differential Geometry of Curves and Surfaces , 1952, Nature.

[2]  Theodosios Pavlidis,et al.  A hierarchical data structure for picture processing , 1975 .

[3]  Charles R. Dyer,et al.  Experiments on Picture Representation Using Regular Decomposition , 1976 .

[4]  Paul Beaudet,et al.  Rotationally invariant image operators , 1978 .

[5]  Ernest L. Hall,et al.  Sequential Hierarchical Scene Matching , 1978, IEEE Transactions on Computers.

[6]  D Marr,et al.  A computational theory of human stereo vision. , 1979, Proceedings of the Royal Society of London. Series B, Biological sciences.

[7]  Donald Bernard Gennery,et al.  Modelling the environment of an exploring vehicle by means of stereo vision , 1980 .

[8]  William B. Thompson,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2009 .

[9]  Shimon Ullman,et al.  Analysis of Visual Motion by Biological and Computer Systems , 1981, Computer.

[10]  P. J. Burt,et al.  Fast Filter Transforms for Image Processing , 1981 .

[11]  P. Burt Fast filter transform for image processing , 1981 .

[12]  John E. W. Mayhew,et al.  Psychophysical and Computational Studies Towards a Theory of Human Stereopsis , 1981, Artif. Intell..

[13]  Azriel Rosenfeld,et al.  Segmentation and Estimation of Image Region Properties through Cooperative Hierarchial Computation , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[14]  J.K. Aggarwal,et al.  Correspondence processes in dynamic scene analysis , 1981, Proceedings of the IEEE.

[15]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[16]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[17]  Allen R. Hanson,et al.  Processing Cones: A Computational Structure for Image Analysis. , 1981 .

[18]  F. Glazer,et al.  Scene Matching by Hierarchical Correlation , 1983 .

[19]  Hans-Hellmut Nagel,et al.  Displacement vectors derived from second-order intensity variations in image sequences , 1983, Comput. Vis. Graph. Image Process..

[20]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Thomas S. Huang,et al.  Some Experiments on Estimating the 3-D Motion Parameters of a Rigid Body from Two Consecutive Image Frames , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[23]  Richard M. Stern,et al.  Fast Computation of the Difference of Low-Pass Transform , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Demetri Terzopoulos,et al.  Multiresolution computation of visible-surface representations , 1984 .

[25]  W. Eric L. Grimson,et al.  Computational Experiments with a Feature Based Stereo Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[27]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[28]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  P. Anandan,et al.  Computing Dense Displacement Fields With Confidence Measures In Scenes Containing Occlusion , 1985, Other Conferences.

[30]  Allen M. Waxman,et al.  Contour Evolution, Neighborhood Deformation, and Global Image Flow: Planar Surfaces in Motion , 1985 .

[31]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[32]  J. van Santen,et al.  Elaborated Reichardt detectors. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[33]  Lance R. Williams,et al.  A Coarse-to-Fine Control Strategy for Stereo and Motion on a , 1986 .

[34]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  F. Glazer Hierarchical Motion Detection , 1987 .

[36]  P. Anandan Measuring Visual Motion From Image Sequences , 1987 .

[37]  Tomaso Poggio,et al.  Probabilistic Solution of Ill-Posed Problems in Computational Vision , 1987 .

[38]  A. Waxman An image flow paradigm , 1987 .

[39]  David J. Heeger,et al.  Optical flow from spatialtemporal filters , 1987 .

[40]  Wilfried Enkelmann,et al.  Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences , 1988, Comput. Vis. Graph. Image Process..

[41]  Hans-Hellmut Nagel,et al.  Image Sequences - Ten (Octal) Years - from phenomenology towards a Theoretical Foundation , 1988, Int. J. Pattern Recognit. Artif. Intell..

[42]  Gilad Adiv,et al.  Inherent Ambiguities in Recovering 3-D Motion and Structure from a Noisy Flow Field , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Edward M. Riseman,et al.  Token-based extraction of straight lines , 1989, IEEE Trans. Syst. Man Cybern..

[44]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.