Attending to Motion: Localizing and Classifying Motion Patterns in Image Sequences

The Selective Tuning Model is a proposal for modelling visual attention in primates and humans. Although supported by significant biological evidence, it is not without its weaknesses. The main one addressed by this paper is that the levels of representation on which it was previously demonstrated (spatial Gaussian pyramids) were not biologically plausible. The motion domain was chosen because enough is known about motion processing to enable a reasonable attempt at defining the feedforward pyramid. The effort is unique because it seems that no past model presents a motion hierarchy plus attention to motion. We propose a neurally-inspired model of the primate visual motion system attempting to explain how a hierarchical feedforward network consisting of layers representing cortical areas V1, MT, MST, and 7a detects and classifies different kinds of motion patterns. The STM model is then integrated into this hierarchy demonstrating that successfully attending to motion patterns, results in localization and labelling of those patterns.

[1]  G. Orban,et al.  Speed and direction selectivity of macaque middle temporal neurons. , 1993, Journal of neurophysiology.

[2]  G. Caputo,et al.  Attentional selection by distractor suppression , 1998, Vision Research.

[3]  Mubarak Shah,et al.  Visual Recognition of Activities, Gestures, Facial Expressions and Speech: An Introduction and a Perspective , 1997 .

[4]  Eileen Kowler,et al.  Attentional interference at small spatial separations , 1999, Vision Research.

[5]  L M Vaina,et al.  Computational modelling of optic flow selectivity in MSTd neurons. , 1998, Network.

[6]  John K. Tsotsos.  Analyzing vision at the complexity level , 1990, Behavioral and Brain Sciences.

[7]  G. Orban,et al.  Velocity sensitivity and direction selectivity of neurons in areas V1 and V2 of the monkey: influence of eccentricity. , 1986, Journal of neurophysiology.

[8]  Hans-Hellmut Nagel,et al.  Tracking Persons in Monocular Image Sequences , 1999, Comput. Vis. Image Underst..

[9]  Sir G. Archaeopteryx.  Object-based attention in the primary visual cortex of the macaque monkey , 1998 .

[10]  R. Desimone,et al.  Responses of Neurons in Inferior Temporal Cortex during Memory- Guided Visual Search , 1998 .

[11]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[13]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[14]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[15]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[16]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[17]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[18]  D. J. Felleman,et al.  Receptive-field properties of neurons in middle temporal visual area (MT) of owl monkeys. , 1984, Journal of neurophysiology.

[19]  T. Meese,et al.  Spiral mechanisms are required to account for summation of complex motion components , 2002, Vision Research.

[20]  R. M. Siegel,et al.  Analysis of optic flow in the monkey parietal area 7a. , 1997, Cerebral cortex.

[21]  John K. Tsotsos.  Motion Understanding: Task-Directed Attention and Representations that Link Perception with Action , 2001, International Journal of Computer Vision.

[22]  R A Andersen,et al.  Neural responses to velocity gradients in macaque cortical area MT , 1996, Visual Neuroscience.

[23]  Constance S. Royden,et al.  Motion perception , 1998 .

[24]  M. Graziano,et al.  Tuning of MST neurons to spiral motions , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  H. Nagel,et al.  On the Selection of Critical Points and Local Curvature Extrema of Region Boundaries for Interframe Matching , 1983 .

[26]  G A Orban,et al.  Attention-dependent suppression of metabolic activity in the early stages of the macaque visual system. , 2000, Cerebral cortex.

[27]  Martin A. Giese.  Neural field model for the recognition of biological motion patterns , 2000 .

[28]  A. T. Smith,et al.  Attentional suppression of activity in the human visual cortex , 2000, Neuroreport.

[29]  T J Sejnowski,et al.  A Model for Encoding Multiple Object Motions and Self-Motion in Area MST of Primate Visual Cortex , 1998, The Journal of Neuroscience.

[30]  John H. R. Maunsell,et al.  Attentional modulation of visual motion processing in cortical areas MT and MST , 1996, Nature.

[31]  Leslie G. Ungerleider,et al.  Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI. , 1998, Science.

[32]  R. Desimone,et al.  Selective attention gates visual processing in the extrastriate cortex. , 1985, Science.

[33]  D. Heeger,et al.  Spatial attention affects brain activity in human primary visual cortex. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Stephen Grossberg,et al.  Neural dynamics of motion integration and segmentation within and across apertures , 2001, Vision Research.

[35]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[36]  John K. Tsotsos,et al.  A framework for visual motion understanding , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[38]  R. Wurtz,et al.  Medial Superior Temporal Area Neurons Respond to Speed Patterns in Optic Flow , 1997, The Journal of Neuroscience.

[39]  Christopher C. Pack,et al.  A Neural Model of Smooth Pursuit Control and Motion Perception by Cortical Area MST , 2001, Journal of Cognitive Neuroscience.

[40]  J A Perrone,et al.  Emulating the Visual Receptive-Field Properties of MST Neurons with a Template Model of Heading Estimation , 1998, The Journal of Neuroscience.

[41]  Claudio S. Pinhanez,et al.  Human action detection using PNF propagation of temporal constraints , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[42]  Jeffrey Mark Siskind,et al.  Grounding language in perception , 1993, Other Conferences.

[43]  Allan D. Jepson,et al.  The Computational Perception of Scene Dynamics , 1997, Comput. Vis. Image Underst..

[44]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[45]  Allan D. Jepson,et al.  Computational Perception of Scene Dynamics , 1996, ECCV.

[46]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[47]  Jake K. Aggarwal,et al.  Human motion analysis: a review , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[48]  Jake K. Aggarwal,et al.  Nonrigid Motion Analysis: Articulated and Elastic Motion , 1998, Comput. Vis. Image Underst..

[49]  Mubarak Shah,et al.  A survey of motion analysis from moving light displays , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[50]  E. DeYoe,et al.  A physiological correlate of the 'spotlight' of visual attention , 1999, Nature Neuroscience.