Computation of optic flow from the motion of edge features in image sequences

Abstract Three-dimensional scene information relating to the depth and orientations of the visible surfaces may be obtained from the optic flow field in time varying imagery. The computation of optic flow is therefore an important step in computer vision. We review our work on calculating optic flow from the motion of edge features in an image sequence. The method is based on a spatiotemporal extension of the Marr-Hildreth edge detection scheme that smooths the data over time as well as over the spatial, image, coordinates. Edge features are defined as the zero crossings of the resultant convolution signal and their motion obtained to subpixel accuracy by a leastsquares interpolation. The details of the method are described and some computational examples are given, including a brief description of how the algorithms may be implemented on a single-instruction multiple-data machine. Some novel effects associated with the choice of metric in the spatiotemporal convolution operator that may be useful for obtaining the time to contact (depth) of objects in the periphery of the field of view are discussed.

[1]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[2]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  D J Hunt,et al.  Distributed processing power in memory , 1983 .

[4]  Peter D. Welch,et al.  The Fast Fourier Transform and Its Applications , 1969 .

[5]  W E Grimson,et al.  A computer implementation of a theory of human stereo vision. , 1981, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[6]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[7]  S. F. Reddaway.  DAP—a distributed array processor , 1973, ISCA 1973.

[8]  J. Canny.  Finding Edges and Lines in Images , 1983 .

[9]  Masahiko Yachida.  Determining velocity maps by spatio-temporal neighborhoods from image sequences , 1983, Comput. Vis. Graph. Image Process..

[10]  K. Nakayama,et al.  Optical Velocity Patterns, Velocity-Sensitive Neurons, and Space Perception: A Hypothesis , 1974, Perception.

[11]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[12]  P. J. Burt,et al.  Fast Filter Transforms for Image Processing , 1981 .

[13]  D. J. Bartliff.  An automatic procedure for image segmentation , 1983, Pattern Recognit. Lett..

[14]  B.L. Yen,et al.  Determining 3-D motion and structure of a rigid body using the spherical projection , 1983, Comput. Vis. Graph. Image Process..

[15]  P. Burt.  Fast filter transform for image processing , 1981 .

[16]  J. E. W. Mayhew,et al.  A computational model of binocular depth perception , 1982, Nature.

[17]  W. R. Moore,et al.  Parallel processor array architecture and its implementation in silicon , 1982 .

[18]  K. Prazdny,et al.  A Simple Method for Recovering a Relative Depth Map in the Case of a Translating Sensor , 1981, IJCAI.

[19]  Dana H. Ballard,et al.  Rigid body motion from depth and optical flow , 1983, Comput. Vis. Graph. Image Process..

[20]  B K Horn,et al.  Calculating the reflectance map. , 1979, Applied optics.

[21]  G Ishai,et al.  Visual stability and space perception in monocular vision: mathematical model. , 1980, Journal of the Optical Society of America.

[22]  F. E. Nicodemus,et al.  Geometrical considerations and nomenclature for reflectance , 1977 .

[23]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[24]  Hans-Hellmut Nagel,et al.  Displacement vectors derived from second-order intensity variations in image sequences , 1983, Comput. Vis. Graph. Image Process..

[25]  William B. Thompson,et al.  Lower-Level Estimation and Interpretation of Visual Motion , 1981, Computer.

[26]  K. Prazdny.  Determining The Instantaneous Direction Of Motion From Optical Flow Generated By A Curvilinearly Moving Observer , 1981, Other Conferences.

[27]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[28]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[29]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[30]  Andrea J. van Doorn,et al.  Invariant Properties of the Motion Parallax Field due to the Movement of Rigid Bodies Relative to an Observer , 1975 .

[31]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[32]  H. Buxton,et al.  Low-level edge detection for the extraction of optical flow data and passive ranging , 1983 .

[33]  B. Buxton,et al.  Monocular depth perception from optical flow by space time signal processing , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[34]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[35]  Thomas S. Huang,et al.  A discrete spatial representation for lateral motion stereo , 1983, Comput. Vis. Graph. Image Process..

[36]  S Prazdny.  A Note on “Perception of Surface Slant and Edge Labels from Optical Flow” , 1981, Perception.

[37]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[38]  R. Paquin,et al.  A spatio-temporal gradient method for estimating the displacement field in time-varying imagery , 1982, Computer Graphics and Image Processing.

[39]  W F Clocksin,et al.  Perception of Surface Slant and Edge Labels from Optical Flow: A Computational Approach , 1980, Perception.

[40]  P. Schönemann.  On artificial intelligence , 1985, Behavioral and Brain Sciences.

[41]  J. Robson.  Spatial and Temporal Contrast-Sensitivity Functions of the Visual System , 1966 .

[42]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[43]  Lee Dn,et al.  The optic flow field: the foundation of vision. , 1980 .

[44]  J Mayhew,et al.  The Interpretation of Stereo-Disparity Information: The Computation of Surface Orientation and Depth , 1982, Perception.

[45]  D. H. Kelly.  Motion and vision. II. Stabilized spatio-temporal threshold surface. , 1979, Journal of the Optical Society of America.

[46]  J. D. Robbins,et al.  Motion-compensated television coding: Part I , 1979, The Bell System Technical Journal.

[47]  Daryl T. Lawton,et al.  Processing translational motion sequences , 1983, Comput. Vis. Graph. Image Process..

[48]  J. D. Robbins,et al.  Motion-compensated coding: Some new results , 1980, The Bell System Technical Journal.