Self-organizing neural networks for perception of visual motion

Abstract The human visual system overcomes ambiguities, collectively known as the aperture problem, in its local measurements of the direction in which visual objects are moving, producing unambiguous percepts of motion. A new approach to the aperture problem is presented, using an adaptive neural network model. The neural network is exposed to moving images during a developmental period and develops its own structure by adapting to statistical characteristics of its visual input history. Competitive learning rules ensure that only connection “chains” between cells of similar direction and velocity sensitivity along successive spatial positions survive. The resultant self-organized configuration implements the type of disambiguation necessary for solving the aperture problem and operates in accord with direction judgments of human experimental subjects. The system not only accommodates its structure to long-term statistics of visual motion, but also simultaneously uses its acquired structure to assimilate, disambiguate, and represent visual motion events in real-time.

[1]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[2]  W. Singer Central control of developmental plasticity in the mammalian visual cortex , 1985, Vision Research.

[3]  John Ross,et al.  Visual processing of motion , 1986, Trends in Neurosciences.

[4]  C. Baker Spatial and temporal determinants of directionally selective velocity preference in cat striate cortex neurons. , 1988, Journal of neurophysiology.

[5]  O. Braddick A short-range process in apparent motion. , 1974, Vision research.

[6]  Stephen Grossberg,et al.  Stereo boundary fusion by cortical complex cells: A system of maps, filters, and feedback networks for multiplexing distributed data , 1989, Neural Networks.

[7]  S. Grossberg,et al.  Neural dynamics of form perception: boundary completion, illusory figures, and neon color spreading. , 1985 .

[8]  S. Grossberg,et al.  ART 2: self-organization of stable category recognition codes for analog input patterns. , 1987, Applied optics.

[9]  D C Van Essen,et al.  Shifter circuits: a computational strategy for dynamic aspects of visual processing. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[10]  H. Barlow Critical limiting factors in the design of the eye and visual cortex , 1981 .

[11]  W. Newsome,et al.  Motion selectivity in macaque visual cortex. II. Spatiotemporal range of directional interactions in MT and V1. , 1986, Journal of neurophysiology.

[12]  S. Grossberg,et al.  How does a brain build a cognitive code? , 1980, Psychological review.

[13]  S. Grossberg Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[14]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[15]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[16]  D. Hubel,et al.  Comparison of the effects of unilateral and bilateral eye closure on cortical unit responses in kittens. , 1965, Journal of neurophysiology.

[17]  Y. Frégnac,et al.  Early development of visual cortical cells in normal and dark‐reared kittens: relationship between orientation selectivity and ocular dominance. , 1978, The Journal of physiology.

[18]  D. Whitteridge,et al.  Connections between pyramidal neurons in layer 5 of cat visual cortex (area 17) , 1987, The Journal of comparative neurology.

[19]  H. Barlow Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[20]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[21]  J. van Santen,et al.  Temporal covariance model of human motion perception. , 1984, Journal of the Optical Society of America. A, Optics and image science.

[22]  V. Ramachandran,et al.  Visual inertia in apparent motion , 1987, Vision Research.

[23]  M. G. Harris,et al.  The perception of moving stimuli: A model of spatiotemporal coding in human vision , 1986, Vision Research.

[24]  Lynn A. Olzak,et al.  Spatial frequency matching in complex stimuli (A) , 1986 .

[25]  J Allman,et al.  Direction- and Velocity-Specific Responses from beyond the Classical Receptive Field in the Middle Temporal Visual Area (MT) , 1985, Perception.

[26]  P. O. Bishop,et al.  End-zone region in receptive fields of hypercomplex and other striate neurons in the cat. , 1979, Journal of neurophysiology.

[27]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[28]  H. Barlow The absolute efficiency of perceptual decisions. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[29]  W. Singer Neuronal Activity as a Shaping Factor in the Self-Organization of Neuron Assemblies , 1983 .

[30]  D. N. Spinelli,et al.  Visual Experience Modifies Distribution of Horizontally and Vertically Oriented Receptive Fields in Cats , 1970, Science.

[31]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[32]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[33]  A. L. Humphrey,et al.  Anatomical banding of intrinsic connections in striate cortex of tree shrews (Tupaia glis) , 1982, The Journal of comparative neurology.

[34]  Teuvo Kohonen,et al.  A Simple Paradigm for the Self-Organized Formation of Structured Feature Maps , 1982 .

[35]  John Tanner,et al.  Integrated optical motion detection , 1986 .

[36]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[37]  John H. R. Maunsell,et al.  Hierarchical organization and functional streams in the visual cortex , 1983, Trends in Neurosciences.

[38]  S. Grossberg Some Psychophysiological and Pharmacological Correlates of A Developmental, Cognitive, and Motivational Theory , 1987 .

[39]  W. Newsome,et al.  Motion selectivity in macaque visual cortex. III. Psychophysics and physiology of apparent motion. , 1986, Journal of neurophysiology.

[40]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation-selective cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[41]  W Gerbino,et al.  Low-Level and High-Level Processes in the Perceptual Organization of Three-Dimensional Apparent Motion , 1984, Perception.

[42]  R. Hetherington The Perception of the Visual World , 1952 .

[43]  W. Greenough,et al.  Experiential modification of the developing brain. , 1975, American scientist.

[44]  W. Newsome,et al.  Motion selectivity in macaque visual cortex. I. Mechanisms of direction and speed selectivity in extrastriate area MT. , 1986, Journal of neurophysiology.

[45]  P. Buisseret,et al.  Role of eye movements in developmental processes of orientation selectivity in the kitten visual cortex , 1986, Vision Research.

[46]  Ellen C Hildreth,et al.  Computing the velocity field along contours , 1986, Workshop on Motion.

[47]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[48]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[49]  P. O. Bishop,et al.  Dimensions and properties of end-zone inhibitory areas in receptive fields of hypercomplex cells in cat striate cortex. , 1979, Journal of neurophysiology.

[50]  J. Gibson The perception of the visual world , 1951 .

[51]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[52]  J. A. Marshall Neural networks for computational vision: motion segmentation and stereo fusion , 1989 .

[53]  Terry Bossomaier,et al.  Why spatial frequency processing in the visual cortex? , 1986, Vision Research.

[54]  J. Gibson,et al.  Parallax and perspective during aircraft landings. , 1955, The American journal of psychology.

[55]  D Kersten,et al.  Predictability and redundancy of natural images. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[56]  A B Watson,et al.  Efficiency of a model human image code. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[57]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[58]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[59]  T. Wiesel,et al.  Functional architecture of macaque monkey visual cortex , 1977 .

[60]  G. Blasdel,et al.  Intrinsic connections of macaque striate cortex: axonal projections of cells outside lamina 4C , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[61]  A. Norcia,et al.  Abnormal acuity development in infantile esotropia. , 1988, Investigative ophthalmology & visual science.

[62]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[63]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[64]  H. Barlow,et al.  The mechanism of directionally selective units in rabbit's retina. , 1965, The Journal of physiology.

[65]  K. Nakayama,et al.  Occlusion and the solution to the aperture problem for motion , 1989, Vision Research.

[66]  M R Rosenzweig,et al.  Effects of differential experience on dendritic spine counts in rat cerebral cortex. , 1973, Journal of comparative and physiological psychology.

[67]  L. Rask ON APPARENT MOVEMENT , 1967 .

[68]  D. Hubel,et al.  Shape and arrangement of columns in cat's striate cortex , 1963, The Journal of physiology.

[69]  H. B. Barlow,et al.  Reconstructing the visual image in space and time , 1979, Nature.

[70]  Y. Frégnac,et al.  Development of neuronal selectivity in primary visual cortex of cat. , 1984, Physiological reviews.

[71]  G. Stent A physiological mechanism for Hebb's postulate of learning. , 1973, Proceedings of the National Academy of Sciences of the United States of America.

[72]  S. Grossberg Processing of expected and unexpected events during conditioning and attention: a psychophysiological theory. , 1982, Psychological review.

[73]  J. Lund,et al.  Intrinsic laminar lattice connections in primate visual cortex , 1983, The Journal of comparative neurology.

[74]  Hugh R. Wilson,et al.  Development of spatiotemporal mechanisms in infant vision , 1988, Vision Research.

[75]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[76]  Jerome A. Feldman,et al.  Time, Space and Form in Vision , 1988 .

[77]  Y Trotter,et al.  The period of susceptibility of visual cortical binocularity to unilateral proprioceptive deafferentation of extraocular muscles. , 1987, Journal of neurophysiology.

[78]  Keiji Tanaka,et al.  Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[79]  T. Kohonen Self-organized formation of topographically correct feature maps , 1982 .

[80]  O J Braddick,et al.  Low-level and high-level processes in apparent motion. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[81]  D. Regan,et al.  Visual processing of four kinds of relative motion , 1986, Vision Research.

[82]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[83]  D. Hubel,et al.  Plasticity of ocular dominance columns in monkey striate cortex. , 1977, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[84]  David J. Fleet,et al.  Spatiotemporal inseparability in early vision: centre‐surround models and velocity selectivity , 1985 .

[85]  G. A. Orban,et al.  Image segregation by motion: cortical mechanisms and implementation in neural networks , 1988 .

[86]  S. Zucker,et al.  Endstopped neurons in the visual cortex as a substrate for calculating curvature , 1987, Nature.

[87]  P. Rakić Prenatal development of the visual system in rhesus monkey. , 1977, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[88]  H. Rodman,et al.  Coding of visual stimulus velocity in area MT of the macaque , 1987, Vision Research.

[89]  J. A. Anderson,et al.  Associative learning of scene parameters from images. , 1987, Applied optics.

[90]  P. Heggelund,et al.  Development of spatial receptive-field organization and orientation selectivity in kitten striate cortex. , 1985, Journal of neurophysiology.

[91]  John D. Pettigrew,et al.  Ultrastructural changes in kitten visual cortex after environmental modification , 1974 .

[92]  S. Grossberg Some Psychophysiological and Pharmacological Correlates of a Developmental, Cognitive and Motivational Theory a , 1984, Annals of the New York Academy of Sciences.

[93]  Ishwar K. Sethi,et al.  Finding Trajectories of Feature Points in a Monocular Image Sequence , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[94]  Y Trotter,et al.  Role of extraocular muscle proprioception in the development of depth perception in cats. , 1987, Journal of neurophysiology.

[95]  S Grossberg,et al.  Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations , 1985, Perception & psychophysics.

[96]  Hans Wallach Über visuell wahrgenommene Bewegungsrichtung , 1935 .

[97]  T. Kohonen Adaptive, associative, and self-organizing functions in neural computing. , 1987, Applied optics.

[98]  J. Lund,et al.  Widespread periodic intrinsic connections in the tree shrew visual cortex. , 1982, Science.

[99]  C. A. Burbeck Position and spatial frequency in large-scale localization judgments , 1987, Vision Research.

[100]  J. Pearson,et al.  Plasticity in the organization of adult cerebral cortical maps: a computer simulation based on neuronal group selection , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[101]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[102]  D. J. Felleman,et al.  Receptive field properties of neurons in area V3 of macaque monkey extrastriate cortex. , 1987, Journal of neurophysiology.

[103]  J. Lund Local circuit neurons of macaque monkey striate cortex: I. Neurons of laminae 4C and 5A , 1987, The Journal of comparative neurology.

[104]  J. Daugman Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[105]  G. Mitchison,et al.  Long axons within the striate cortex: their distribution, orientation, and patterns of connection. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[106]  T. Pasternak,et al.  Pattern and motion vision in cats with selective loss of cortical directional selectivity , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[107]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[108]  S. Grossberg How does a brain build a cognitive code , 1980 .

[109]  S M Archer,et al.  A role for action-potential activity in the development of neuronal connections in the kitten retinogeniculate pathway , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[110]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[111]  T. Albright Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[112]  S Grossberg,et al.  Masking fields: a massively parallel neural architecture for learning, recognizing, and predicting multiple groupings of patterned data. , 1987, Applied optics.

[113]  P. Leese Comparison of the effects , 1992 .

[114]  G Sperling,et al.  Three theories of stroboscopic motion detection. , 1985, Spatial vision.

[115]  G. Phillips,et al.  Cooperative phenomena in the perception of motion direction. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[116]  J. Lund,et al.  Local circuit neurons of macaque monkey striate cortex: III. Neurons of laminae 4B, 4A, and 3B , 1997, The Journal of comparative neurology.

[117]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[118]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation columns. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[119]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[120]  J. Duysens,et al.  Response properties of area 17 neurons in cats reared in stroboscopic illumination. , 1987, Journal of neurophysiology.

[121]  M. Sanders Handbook of Sensory Physiology , 1975 .

[122]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[123]  K. Tanaka,et al.  Analysis of discontinuity in visual contours in area 19 of the cat , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[124]  G A Orban,et al.  Response properties of visual cortical neurons in cats reared in stroboscopic illumination. , 1983, Journal of neurophysiology.

[125]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[126]  K. Nakayama,et al.  Optical Velocity Patterns, Velocity-Sensitive Neurons, and Space Perception: A Hypothesis , 1974, Perception.

[127]  R N Weinreb,et al.  The cytoskeleton of the cultured human trabecular cell. Characterization and drug responses. , 1988, Investigative ophthalmology & visual science.

[128]  R. McCann On absolute stability , 1972 .

[129]  Allen M. Waxman,et al.  Binocular Image Flows: Steps Toward Stereo-Motion Fusion , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[130]  T. Wiesel,et al.  Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[131]  D. Hubel,et al.  The period of susceptibility to the physiological effects of unilateral eye closure in kittens , 1970, The Journal of physiology.

[132]  P. O. Bishop,et al.  Hypercomplex and simple/complex cell classifications in cat striate cortex. , 1978, Journal of neurophysiology.

[133]  E. Adelson,et al.  Phenomenal coherence of moving visual patterns , 1982, Nature.