A Theory of Geometric Constraints on Neural Activity for Natural Three-Dimensional Movement

Although the orientation of an arm in space or the static view of an object may be represented by a population of neurons in complex ways, how these variables change with movement often follows simple linear rules, reflecting the underlying geometric constraints in the physical world. A theoretical analysis is presented for how such constraints affect the average firing rates of sensory and motor neurons during natural movements with low degrees of freedom, such as a limb movement and rigid object motion. When applied to nonrigid reaching arm movements, the linear theory accounts for cosine directional tuning with linear speed modulation, predicts a curl-free spatial distribution of preferred directions, and also explains why the instantaneous motion of the hand can be recovered from the neural population activity. For three-dimensional motion of a rigid object, the theory predicts that, to a first approximation, the response of a sensory neuron should have a preferred translational direction and a preferred rotation axis in space, both with cosine tuning functions modulated multiplicatively by speed and angular speed, respectively. Some known tuning properties of motion-sensitive neurons follow as special cases. Acceleration tuning and nonlinear speed modulation are considered in an extension of the linear theory. This general approach provides a principled method to derive mechanism-insensitive neuronal properties by exploiting the inherently low dimensionality of natural movements.

[1]  A. Schwartz,et al.  Motor cortical activity during drawing movements: population representation during sinusoid tracing. , 1993, Journal of neurophysiology.

[2]  A. P. Georgopoulos,et al.  Primate motor cortex and free arm movements to visual targets in three- dimensional space. I. Relations between single cell discharge and direction of movement , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: anatomic location and visual response properties. , 1993, Journal of neurophysiology.

[4]  J. Kalaska,et al.  Tactile activity in primate primary somatosensory cortex during active arm movements: correlation with receptive field properties. , 1994, Journal of neurophysiology.

[5]  A. P. Georgopoulos,et al.  Primate motor cortex and free arm movements to visual targets in three- dimensional space. II. Coding of the direction of movement by a neuronal population , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  M. E. Anderson,et al.  Pallidal discharge related to the kinematics of reaching movements in two dimensions. , 1997, Journal of neurophysiology.

[7]  S L Moody,et al.  A Model That Accounts for Activity in Primate Frontal Cortex during a Delayed Matching-to-Sample Task , 1998, The Journal of Neuroscience.

[8]  A P Georgopoulos,et al.  Motor cortical activity in a context-recall task. , 1995, Science.

[9]  H. Flanders Differential Forms with Applications to the Physical Sciences , 1964 .

[10]  P. Viviani,et al.  The law relating the kinematic and figural aspects of drawing movements. , 1983, Acta psychologica.

[11]  C. P. Watson,et al.  Analysis of Motion , 1921 .

[12]  E. Bizzi,et al.  Neural, mechanical, and geometric factors subserving arm posture in humans , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Allan D. Jepson,et al.  Visual Perception of Three-Dimensional Motion , 1990, Neural Computation.

[15]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[16]  R A Andersen,et al.  The Analysis of Complex Motion Patterns by Form/Cue Invariant MSTd Neurons , 1996, The Journal of Neuroscience.

[17]  F. Mussa-Ivaldi,et al.  Do neurons in the motor cortex encode movement direction? An alternative hypothesis , 1988, Neuroscience Letters.

[18]  G. E. Alexander,et al.  Neural correlates of a spatial sensory-to-motor transformation in primary motor cortex. , 1997, Journal of neurophysiology.

[19]  Margaret E. Sereno,et al.  Learning to See Rotation and Dilation with a Hebb Rule , 1990, NIPS.

[20]  Hideo Sakata,et al.  Functional properties of rotation-sensitive neurons in the posterior parietal association cortex of the monkey , 1994, Experimental Brain Research.

[21]  S. Edelman,et al.  Orientation dependence in the recognition of familiar and novel views of three-dimensional objects , 1992, Vision Research.

[22]  J Allman,et al.  Direction- and Velocity-Specific Responses from beyond the Classical Receptive Field in the Middle Temporal Visual Area (MT) , 1985, Perception.

[23]  D. Heeger Modeling simple-cell direction selectivity with normalized, half-squared, linear operators. , 1993, Journal of neurophysiology.

[24]  B. Peterson,et al.  Optimal response planes and canal convergence in secondary neurons in vestibular nuclei of alert cats , 1984, Brain Research.

[25]  D. Zipser,et al.  A Model of Reaching Dynamics in Primary Motor Cortex , 1998, Journal of Cognitive Neuroscience.

[26]  G E Alexander,et al.  Neural representations of the target (goal) of visually guided arm movements in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[27]  Kechen Zhang,et al.  Emergence of Position-Independent Detectors of Sense of Rotation and Dilation with Hebbian Learning: An Analysis , 1999, Neural Computation.

[28]  G Gaál Calculation of movement direction from firing activities of neurons in intrinsic co-ordinate systems defined by their preferred directions. , 1993, Journal of theoretical biology.

[29]  A. Verri,et al.  Green Theorems and Qualitative Properties of the Optical Flow , 1991 .

[30]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[31]  G E Alexander,et al.  Preferential representation of instructed target location versus limb trajectory in dorsal premotor area. , 1997, Journal of neurophysiology.

[32]  R A Andersen,et al.  Transparent motion perception as detection of unbalanced motion signals. III. Modeling , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  S Thorpe,et al.  Modulation of neural stereoscopic processing in primate area V1 by the viewing distance. , 1992, Science.

[34]  J. Kalaska,et al.  Cerebellar neuronal activity related to whole-arm reaching movements in the monkey. , 1989, Journal of neurophysiology.

[35]  T. Albright Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[36]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[37]  Terence D. Sanger,et al.  Theoretical Considerations for the Analysis of Population Coding in Motor Cortex , 1994, Neural Computation.

[38]  T D Albright,et al.  Form-cue invariant motion processing in primate visual cortex. , 1992, Science.

[39]  K. Tanaka,et al.  Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  J. Kalaska,et al.  Proprioceptive activity in primate primary somatosensory cortex during active arm reaching movements. , 1994, Journal of neurophysiology.

[41]  C A Buneo,et al.  Postural Dependence of Muscle Actions: Implications for Neural Control , 1997, The Journal of Neuroscience.

[42]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[43]  田中 啓治 Analysis of Local and Wide-Field Movements in the Superior Temporal Visual Areas of the Macaque Monkey , 1987 .

[44]  A B Schwartz,et al.  Direct cortical representation of drawing. , 1994, Science.

[45]  R. Andersen,et al.  Transparent motion perception as detection of unbalanced motion signals. II. Physiology , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[46]  Peter E. Latham,et al.  Statistically Efficient Estimation Using Population Coding , 1998, Neural Computation.

[47]  G. Orban,et al.  Shape and Spatial Distribution of Receptive Fields and Antagonistic Motion Surrounds in the Middle Temporal Area (V5) of the Macaque , 1995, The European journal of neuroscience.

[48]  S. Scott,et al.  Reaching movements with similar hand paths but different arm orientations. I. Activity of individual cells in motor cortex. , 1997, Journal of neurophysiology.

[49]  Alan Watt,et al.  Advanced animation and rendering techniques - theory and practice , 1992 .

[50]  L. Abbott,et al.  A model of multiplicative neural responses in parietal cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[51]  T. Ebner,et al.  Neuronal specification of direction and distance during reaching movements in the superior precentral premotor area and primary motor cortex of monkeys. , 1993, Journal of neurophysiology.

[52]  J. W. Humberston Classical mechanics , 1980, Nature.

[53]  R. Wurtz,et al.  Planar directional contributions to optic flow responses in MST neurons. , 1997, Journal of neurophysiology.

[54]  J Duysens,et al.  Neurons in the ventral intraparietal area of awake macaque monkey closely resemble neurons in the dorsal part of the medial superior temporal area in their responses to optic flow patterns. , 1996, Journal of neurophysiology.

[55]  H. Rodman,et al.  Coding of visual stimulus velocity in area MT of the macaque , 1987, Vision Research.

[56]  M. Flanders,et al.  Two components of muscle activation: scaling with the speed of arm movement. , 1992, Journal of neurophysiology.

[57]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[58]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[59]  H. Komatsu,et al.  Disparity sensitivity of neurons in monkey extrastriate area MST , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  R. M. Siegel,et al.  Analysis of optic flow in the monkey parietal area 7a. , 1997, Cerebral cortex.

[61]  E. Fetz Movement control: Are movement parameters recognizably coded in the activity of single neurons? , 1992 .

[62]  Leslie G. Ungerleider Two cortical visual systems , 1982 .

[63]  J. Kalaska,et al.  A comparison of movement direction-related versus load direction- related activity in primate motor cortex, using a two-dimensional reaching task , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[64]  K. Tanaka,et al.  Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. , 1989, Journal of neurophysiology.

[65]  A. P. Georgopoulos,et al.  Movement parameters and neural activity in motor cortex and area 5. , 1994, Cerebral cortex.

[66]  G. Orban,et al.  The Speed Tuning of Medial Superior Temporal (Mst) Cell Responses to Optic-Flow Components , 1995, Perception.

[67]  J A Movshon,et al.  Visual cortical signals supporting smooth pursuit eye movements. , 1990, Cold Spring Harbor Symposia on Quantitative Biology.

[68]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[69]  Keiji Tanaka,et al.  Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[70]  Jan J. Koenderink,et al.  Local structure of movement parallax of the plane , 1976 .

[71]  J. F. Soechting,et al.  Moving effortlessly in three dimensions: does Donders' law apply to arm movement? , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[72]  H. C. Corben,et al.  Classical Mechanics (2nd ed.) , 1961 .

[73]  T J Sejnowski,et al.  A Model for Encoding Multiple Object Motions and Self-Motion in Area MST of Primate Visual Cortex , 1998, The Journal of Neuroscience.

[74]  James V. Stone Object recognition using spatiotemporal signatures , 1998, Vision Research.

[75]  John H. R. Maunsell,et al.  Attentional modulation of visual motion processing in cortical areas MT and MST , 1996, Nature.

[76]  K. Tanaka,et al.  Comparison of neuronal selectivity for stimulus speed, length, and contrast in the prestriate visual cortical areas V4 and MT of the macaque monkey. , 1994, Journal of neurophysiology.

[77]  J. A. Pruszynski,et al.  Neural correlates , 2023 .

[78]  G. Miller,et al.  The Role of Learning. , 1951 .

[79]  Thomas D. Albright,et al.  Neural correlates of perceptual motion coherence , 1992, Nature.

[80]  David C. Knill,et al.  Object classification for human and ideal observers , 1995, Vision Research.

[81]  J. Perrone,et al.  A model of self-motion estimation within primate extrastriate visual cortex , 1994, Vision Research.

[82]  H. Komatsu,et al.  Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. , 1988, Journal of neurophysiology.

[83]  M. Graziano,et al.  Tuning of MST neurons to spiral motions , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[84]  J F Kalaska,et al.  Systematic changes in directional tuning of motor cortex cell activity with hand location in the workspace during generation of static isometric forces in constant spatial directions. , 1997, Journal of neurophysiology.

[85]  B L McNaughton,et al.  Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. , 1998, Journal of neurophysiology.

[86]  T. Albright,et al.  Contribution of area MT to perception of three-dimensional shape: a computational study , 1996, Vision Research.

[87]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[88]  D. Zipser,et al.  Identification models of the nervous system , 1992, Neuroscience.

[89]  S. Squatrito,et al.  Encoding of Smooth Pursuit Direction and Eye Position by Neurons of Area MSTd of Macaque Monkey , 1997, The Journal of Neuroscience.

[90]  P. Thier,et al.  Responses of Visual‐Tracking Neurons from Cortical Area MST‐I to Visual, Eye and Head Motion , 1992, The European journal of neuroscience.

[91]  J. T. Massey,et al.  Mental rotation of the neuronal population vector. , 1989, Science.

[92]  S. Zeki Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey , 1974, The Journal of physiology.

[93]  S P Wise,et al.  Primate premotor cortex: dissociation of visuomotor from sensory signals. , 1992, Journal of neurophysiology.

[94]  A. Schwartz,et al.  Motor cortical activity during drawing movements: population representation during lemniscate tracing. , 1999 .

[95]  Kang Cheng,et al.  Neuronal selectivity for stimulus length in the prestriate areas V4 and MT of the Macaque visual cortex , 1991 .

[96]  N. Logothetis,et al.  Psychophysical and physiological evidence for viewer-centered object representations in the primate. , 1995, Cerebral cortex.

[97]  M. Flanders,et al.  Arm muscle activation for static forces in three-dimensional space. , 1990, Journal of neurophysiology.

[98]  A. P. Georgopoulos,et al.  Primate motor cortex and free arm movements to visual targets in three- dimensional space. III. Positional gradients and population coding of movement direction from various movement origins , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[99]  A. Georgopoulos,et al.  The motor cortex and the coding of force. , 1992, Science.

[100]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[101]  B W Peterson,et al.  Sensorimotor transformation in the cat's vestibuloocular reflex system. I. Neuronal signals coding spatial coordination of compensatory eye movements. , 1993, Journal of neurophysiology.

[102]  Ruye Wang A Simple Competitive Account of Some Response Properties of Visual Neurons in Area MSTd , 1995, Neural Computation.

[103]  J. Kalaska,et al.  Comparison of cerebellar and motor cortex activity during reaching: directional tuning and response variability. , 1993, Journal of neurophysiology.

[104]  M Lappe,et al.  Neuronal responses in the motion pathway of the macaque monkey to natural optic flow stimuli , 1996, Neuroreport.

[105]  Paul B. Johnson,et al.  Making arm movements within different parts of space: dynamic aspects in the primate motor cortex , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[106]  Margaret E. Sereno Neural Computation of Pattern Motion: Modeling Stages of Motion Analysis in the Primate Visual Cortex , 1993 .

[107]  A B Schwartz,et al.  Motor cortical representation of speed and direction during reaching. , 1999, Journal of neurophysiology.

[108]  H H Bülthoff,et al.  How are three-dimensional objects represented in the brain? , 1994, Cerebral cortex.

[109]  R. Wurtz,et al.  Medial Superior Temporal Area Neurons Respond to Speed Patterns in Optic Flow , 1997, The Journal of Neuroscience.

[110]  Tomaso Poggio,et al.  Role of learning in three-dimensional form perception , 1996, Nature.

[111]  Keiji Tanaka,et al.  Optical Imaging of Functional Organization in the Monkey Inferotemporal Cortex , 1996, Science.

[112]  R Caminiti,et al.  Making arm movements within different parts of space: the premotor and motor cortical representation of a coordinate system for reaching to visual targets , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[113]  A. Dobbins,et al.  Distance modulation of neural activity in the visual cortex. , 1998, Science.

[114]  F. A. Mussa-lvaldi,et al.  Convergent force fields organized in the frog's spinal cord , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[115]  R. Wurtz,et al.  Response to motion in extrastriate area MSTl: center-surround interactions. , 1998, Journal of neurophysiology.

[116]  R. Wurtz,et al.  Responses of MT and MST neurons to one and two moving objects in the receptive field. , 1997, Journal of neurophysiology.

[117]  R. Andersen,et al.  Mechanisms of Heading Perception in Primate Visual Cortex , 1996, Science.

[118]  K. Hoffmann,et al.  Optic Flow Processing in Monkey STS: A Theoretical and Experimental Approach , 1996, The Journal of Neuroscience.

[119]  E. Adelson,et al.  The analysis of moving visual patterns , 1985 .

[120]  R. Wurtz,et al.  Response of monkey MST neurons to optic flow stimuli with shifted centers of motion , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.