Biological computation of image motion from flows over boundaries

A theory of early motion processing in the human and primate visual system is presented which is based on the idea that spatio-temporal retinal image data is represented in primary visual cortex by a truncated 3D Taylor expansion that we refer to as a jet vector. This representation allows all the concepts of differential geometry to be applied to the analysis of visual information processing. We show in particular how the generalised Stokes theorem can be used to move from the calculation of derivatives of image brightness at a point to the calculation of image brightness differences on the boundary of a volume in space-time and how this can be generalised to apply to integrals of products of derivatives. We also provide novel interpretations of the roles of direction selective, bi-directional and pan-directional cells and of type I and type II cells in V5/MT.

[1]  Richard A. Young,et al.  Physiological model of motion analysis for machine vision , 1993, Electronic Imaging.

[2]  M. J. Wright,et al.  Matching velocity in central and peripheral vision , 1986, Vision Research.

[3]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[4]  R. Desimone,et al.  Columnar organization of directionally selective cells in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[5]  P. Thompson.  Perceived rate of movement depends on contrast , 1982, Vision Research.

[6]  Hans-Hellmut Nagel,et al.  Estimation of Optical Flow Based on Higher-Order Spatiotemporal Derivatives in Interlaced and Non-Interlaced Image Sequences , 1995, Artif. Intell..

[7]  Patrick Bouthemy,et al.  Computation and analysis of image motion: A synopsis of current problems and methods , 1996, International Journal of Computer Vision.

[8]  Andrea J. van Doorn,et al.  Receptive field assembly pattern specificity , 1992, J. Vis. Commun. Image Represent..

[9]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[10]  A Johnston,et al.  Concurrent measurement of perceived speed and speed discrimination threshold using the method of single stimuli , 1999, Vision Research.

[11]  D. Hubel,et al.  Sequence regularity and geometry of orientation columns in the monkey striate cortex , 1974, The Journal of comparative neurology.

[12]  C P Benton,et al.  Induced motion at texture–defined motion boundaries , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[13]  A. Verri,et al.  Computational aspects of motion perception in natural and artificial vision systems. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[14]  W. Newsome,et al.  Motion selectivity in macaque visual cortex. I. Mechanisms of direction and speed selectivity in extrastriate area MT. , 1986, Journal of neurophysiology.

[15]  Charles G. Gross,et al.  Pattern recognition mechanisms , 1985 .

[16]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[17]  Chuan Yi Tang,et al.  A 2.|E|-Bit Distributed Algorithm for the Directed Euler Trail Problem , 1993, Inf. Process. Lett..

[18]  D. H. Kelly.  Visual responses to time-dependent stimuli. III. Individual variations. , 1962, Journal of the Optical Society of America.

[19]  Jan J. Koenderink,et al.  Solid shape , 1990 .

[20]  G. Orban,et al.  Response to movement of neurons in areas 17 and 18 of the cat: velocity sensitivity. , 1981, Journal of neurophysiology.

[21]  K. Tanaka,et al.  Comparison of neuronal selectivity for stimulus speed, length, and contrast in the prestriate visual cortical areas V4 and MT of the macaque monkey. , 1994, Journal of neurophysiology.

[22]  J. Koenderink,et al.  Extraction of motion parallax structure in the visual system I , 2004, Biological Cybernetics.

[23]  E. Adelson,et al.  Phenomenal coherence of moving visual patterns , 1982, Nature.

[24]  C. Baker,et al.  Residual motion perception in a "motion-blind" patient, assessed with limited-lifetime random dot stimuli , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  J. Koenderink,et al.  Representation of local geometry in the visual system , 1987, Biological Cybernetics.

[26]  M. Srinivasan.  Generalized gradient schemes for the measurement of two-dimensional image motion , 1990, Biological Cybernetics.

[27]  Steven H. Weintraub,et al.  Differential Forms: A Complement to Vector Calculus , 1997 .

[28]  J. J. Koenderink,et al.  Extraction of motion parallax structure in the visual system II , 1990, Biological Cybernetics.

[29]  P. McOwan,et al.  A computational model of the analysis of some first-order and second-order motion patterns by simple and complex cells , 1992, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[30]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[31]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[32]  P. McLeod,et al.  Preserved and Impaired Detection of Structure From Motion by a 'Motion-blind" Patient , 1996 .

[33]  Laveen N. Kanal,et al.  Gradient Based Image Motion Estimation Without Computing Gradients , 1997, International Journal of Computer Vision.

[34]  J. Koenderink.  Operational significance of receptive field assemblies , 1988, Biological Cybernetics.

[35]  Peter W. McOwan,et al.  Robust velocity computation from a biologically motivated model of motion perception , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[36]  G. Orban,et al.  Speed and direction selectivity of macaque middle temporal neurons. , 1993, Journal of neurophysiology.

[37]  C P Benton,et al.  A new approach to analysing texture-defined motion , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[38]  A. Johnston,et al.  A unified account of three apparent motion illusions , 1995, Vision Research.

[39]  D. H. Kelly.  Visual response to time-dependent stimuli. I. Amplitude sensitivity measurements. , 1961, Journal of the Optical Society of America.

[40]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[41]  G. Orban,et al.  Processing of kinetically defined boundaries in the cortical motion area MT of the macaque monkey. , 1995, Journal of neurophysiology.

[42]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[43]  Mark A. Georgeson,et al.  Visual perception : physiology, psychology, & ecology , 2003 .

[44]  Michael Jenkin,et al.  Spatial vision in humans and robots , 1994 .

[45]  Toronto.  Spatial Vision in Humans and Robots , 1995, Neurology.

[46]  H. Rodman,et al.  Coding of visual stimulus velocity in area MT of the macaque , 1987, Vision Research.