An architectural model of visual motion understanding

The past few years have seen an explosion of interest in the recovery and use of visual motion information by biological and machine vision systems. In the area of computer vision, a variety of algorithms have been developed for extracting various types of motion information from images. Neuroscientists have made great strides in understanding the flow of motion information from the retina to striate and extrastriate cortex. The psychophysics community has gone a long way toward characterizing the limits and structure of human motion processing. The central claim of this thesis is that many puzzling aspects of motion perception can be understood by assuming a particular architecture for the human motion processing system. The architecture consists of three functional units or subsystems. The first or low-level subsystem computes simple mathematical properties of the visual signal. It is entirely bottom-up, and prone to error when its implicit assumptions are violated. The intermediate-level subsystem combines the low-level system's output with world knowledge, segmentation information and other inputs to construct a representation of the world in terms of primitive forms and their trajectories. It is claimed to be the substrate for long-range apparent motion. The highest level of the motion system assembles intermediate level form and motion primitives into scenarios that can be used for prediction and for matching against stored models. The architecture described above is the result of joint work with Jerome Feldman and Nigel Goddard (Feldman, 1988). The description of the low-level system is in accord with the standard view of early motion processing, and the details of the high-level system are being worked out in (Goddard). The secondary contribution of this thesis is a detailed connectionist model of the intermediate level of the architecture. In order to compute the trajectories of primitive shapes it is necessary to design mechanisms for handling time and Gestalt grouping effects in connectionist networks. Solutions to these problems are developed and used to construct a network that interprets continuous and apparent motion stimuli in a limited domain. Simulation results show that its interpretations are in qualitative agreement with human perception.

[1]  L Maffei,et al.  Spatial‐frequency characteristics of neurones of area 18 in the cat: dependence on the velocity of the visual stimulus. , 1985, The Journal of physiology.

[2]  T. Poggio,et al.  A synaptic mechanism possibly underlying directional selectivity to motion , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[3]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[4]  Marc Green,et al.  Inhibition and facilitation of apparent motion by real motion , 1983, Vision Research.

[5]  R. Gregory.  The intelligent eye , 1970 .

[6]  Steven W. Zucker,et al.  A Gradient Projection Algorithm for Relaxation Methods , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[8]  E. Hildreth.  The computation of the velocity field , 1984, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[9]  W. Newsome,et al.  Deficits in visual motion processing following ibotenic acid lesions of the middle temporal visual area of the macaque monkey , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  Alan L. Yuille,et al.  The Motion Coherence Theory , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[11]  Dana H. Ballard,et al.  Cortical connections and parallel processing: Structure and function , 1986, Behavioral and Brain Sciences.

[12]  John A. Baro,et al.  Apparent motion can be perceived between patterns with dissimilar spatial frequencies , 1988, Vision Research.

[13]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[14]  M W von Grünau,et al.  The involvement of illusory contours in stroboscopic motion. , 1979, Perception & psychophysics.

[15]  Ken Nakayama,et al.  Biological image motion processing: A review , 1985, Vision Research.

[16]  Ruud M. Bolle,et al.  Generalized neighborhoods: a new approach to complex parameter feature extraction , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  J. Feldman.  Four frames suffice: A provisional model of vision and space , 1985, Behavioral and Brain Sciences.

[18]  G Johansson,et al.  Spatio-temporal differentiation and integration in visual motion perception , 1976, Psychological research.

[19]  W. Newsome,et al.  Motion selectivity in macaque visual cortex. I. Mechanisms of direction and speed selectivity in extrastriate area MT. , 1986, Journal of neurophysiology.

[20]  Bruno G. Breitmeyer,et al.  The role of visual pattern persistence in bistable stroboscopic motion , 1986, Vision Research.

[21]  Andrew M. Derrington,et al.  Errors in direction-of-motion discrimination with complex stimuli , 1987, Vision Research.

[22]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[23]  David N. Lee.  Visual proprioceptive control of stance , 1975 .

[24]  Geoffrey E. Hinton.  Shape Representation in Parallel Systems , 1981, IJCAI.

[25]  V. Ramachandran,et al.  Displacement thresholds for coherent apparent motion in random dot-patterns , 1983, Vision Research.

[26]  B. C. Motter,et al.  The functional properties of the light-sensitive neurons of the posterior parietal cortex studied in waking monkeys: foveal sparing and opponent vector organization , 1981, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  R. F. Rashid,et al.  Towards a system for the interpretation of moving light displays , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  D. Regan,et al.  Separable aftereffects of changing-size and motion-in-depth: Different neural mechanisms? , 1979, Vision Research.

[29]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[30]  A. Pantle,et al.  On the mechanism that encodes the movement of contrast variations: Velocity discrimination , 1989, Vision Research.

[31]  Jerome A. Feldman,et al.  Time, Space and Form in Vision , 1988 .

[32]  I. Rock.  The Logic of Perception , 1983 .

[33]  D. Regan,et al.  Evidence for the existence of neural mechanisms selectively sensitive to the direction of movement in space , 1973, The Journal of physiology.

[34]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  D Regan,et al.  Visual responses to vorticity and the neural analysis of optic flow. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[36]  Nigel Goddard,et al.  Rochester Connectionist Simulator , 1989 .

[37]  V. Ramachandran,et al.  Apparent movement with subjective contours. , 1973, Vision research.

[38]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[39]  Stuart Anstis,et al.  Entrained path deflection in apparent motion , 1986, Vision Research.

[40]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[41]  Dana H. Ballard,et al.  Parameter Networks: Towards a Theory of Low-Level Vision , 1981, IJCAI.

[42]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[43]  P. Burt,et al.  Time, distance, and feature trade-offs in visual apparent motion. , 1981, Psychological review.

[44]  E C Hildreth,et al.  Incremental rigidity scheme for recovering structure from motion: position-based versus velocity-based formulations. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[45]  Jian Wu,et al.  Computing visual motion in the short and the long: from receptive fields to neural networks , 1989, [1989] Proceedings. Workshop on Visual Motion.

[46]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[47]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[48]  C. Koch,et al.  The analysis of visual motion: from computational theory to neuronal mechanisms. , 1986, Annual review of neuroscience.

[49]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[50]  J. T. Petersik,et al.  Failure to find an absolute retinal limit of a putative short-range process in apparent motion , 1983, Vision Research.

[51]  Alex Pentland,et al.  Perceptual Organization and the Representation of Natural Form , 1986, Artif. Intell..

[52]  P. A. Kolers.  Aspects of motion perception , 1972 .

[53]  A. Pantle.  Motion aftereffect magnitude as a measure of the spatio-temporal response properties of direction-sensitive analyzers. , 1974, Vision research.

[54]  P. A. Kolers,et al.  Shape and color in apparent motion , 1976, Vision Research.

[55]  J. Farrell,et al.  Visual transformations underlying apparent movement , 1983, Perception & psychophysics.

[56]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[57]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[58]  K Prazdny,et al.  What Variables Control (Long-Range) Apparent Motion? , 1986, Perception.

[59]  T. Albright.  Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[60]  G. Wasilkowski,et al.  Computing optical flow , 1989, [1989] Proceedings. Workshop on Visual Motion.

[61]  L Chen,et al.  Topological Structure in the Perception of Apparent Motion , 1985, Perception.

[62]  R. Sekuler,et al.  The independence of channels in human vision selective for direction of movement. , 1975, The Journal of physiology.

[63]  Suzanne P. McKee,et al.  Is there a constancy for velocity? , 1989, Vision Research.

[64]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  C. A. Burbeck,et al.  Further evidence for a broadband, isotropic mechanism sensitive to high-velocity stimuli , 1987, Vision Research.

[66]  R N Shepard,et al.  Path-guided apparent motion. , 1983, Science.

[67]  William B. Thompson,et al.  Introduction to the Special Issue on Visual Motion , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[68]  R. Mansfield,et al.  Analysis of visual behavior , 1982 .

[69]  J. E. Brown,et al.  Brown , 1975 .

[70]  B. Julesz.  Foundations of Cyclopean Perception , 1971 .

[71]  Robert K. Cunningham,et al.  The Neural Analog Diffusion-Enhancement Layer (NADEL) And Early Visual Processing , 1988, Other Conferences.

[72]  R. Sekuler,et al.  Aftereffect of Seen Motion with a Stabilized Retinal Image , 1963, Science.

[73]  Leslie G. Ungerleider.  Two cortical visual systems , 1982 .

[74]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[75]  O J Braddick,et al.  Low-level and high-level processes in apparent motion. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[76]  Lokendra Shastri,et al.  Evidential Reasoning in Semantic Networks: A Formal Theory , 1985, IJCAI.

[77]  V. S. Ramachandran,et al.  Perception of illusory occlusion in apparent motion , 1986, Vision Research.

[78]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[79]  S. McKee,et al.  Precise velocity discrimination despite random variations in temporal frequency and contrast , 1986, Vision Research.

[80]  David J. Heeger,et al.  Optical flow from spatialtemporal filters , 1987 .

[81]  Demetri Terzopoulos,et al.  Regularization of Inverse Visual Problems Involving Discontinuities , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.