Chapter 23 Information transfer between sensory and motor networks

Publisher Summary This chapter focuses on the information transfer between sensory and motor networks. The sensory and motor structures of the brain encode information in various different formats. Sensorimotor transformations interface the sensory and motor systems. They have two main tasks to solve. First, they must convert between the different coding formats. Second, they must fuse the different sensory and motor input signals and establish a unified representation of the environment and the organism's action within it. The fibers of the retinal ganglion cells form a thick parallel bundle, called the “optic tract,” from which branches run to several parallel retinal recipient structures. The main cortical pathway is via the thalamus to the primary visual cortex, or area V1. Other important pathways are those to the superior colliculus and to the accessory optic system and pretectum. Each retinal ganglion cell transmits information from a small-localized part of the visual image, known as its “receptive field.” The chapter discusses the position of the limbs and the activity of the muscles; this is known as “proprioceptive information” and is generated by spindle fibers in the muscles.

[1]  F Bremmer,et al.  Eye position effects on the neuronal activity of dorsal premotor cortex in the macaque monkey. , 1998, Journal of neurophysiology.

[2]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[3]  F. Bremmer,et al.  Spatial invariance of visual receptive fields in parietal cortex neurons , 1997, Nature.

[4]  L F Abbott,et al.  Transfer of coded information from sensory to motor networks , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  S. Thorpe,et al.  Neural processing of stereopsis as a function of viewing distance in primate visual cortical area V1 , 1996 .

[6]  R. M. Siegel,et al.  Perception of three-dimensional structure from motion in monkey and man , 1988, Nature.

[7]  Ning Qian,et al.  Computing Stereo Disparity and Motion with Known Binocular Cell Properties , 1994, Neural Computation.

[8]  Richard A. Andersen,et al.  Recovering three-dimensional structure from motion with surface reconstruction , 1991, Vision Research.

[9]  A Berthoz,et al.  A neural network model of sensoritopic maps with predictive short-term memory properties. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[10]  C. Duffy MST neurons respond to optic flow and translational movement. , 1998, Journal of neurophysiology.

[11]  Charles G. Gross,et al.  Pattern recognition mechanisms , 1985 .

[12]  B. McNaughton,et al.  Perception, memory, and emotion : frontiers in neuroscience , 1996 .

[13]  E. Bizzi,et al.  Posture control and trajectory formation during arm movement , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  W K Page,et al.  MST neuronal responses to heading direction during pursuit eye movements. , 1999, Journal of neurophysiology.

[15]  David C. Burr,et al.  Compression of visual space before saccades , 1997, Nature.

[16]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[17]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[18]  J. Schall,et al.  Neural selection and control of visually guided eye movements. , 1999, Annual review of neuroscience.

[19]  L. Fogassi,et al.  Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[20]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  T. Flash,et al.  Moving gracefully: quantitative theories of motor coordination , 1987, Trends in Neurosciences.

[22]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[23]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[24]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[25]  J. Gibson The perception of the visual world , 1951 .

[26]  Ellen C. Hildreth,et al.  Recovering heading for visually-guided navigation , 1992, Vision Research.

[27]  D A Robinson,et al.  The use of control systems analysis in the neurophysiology of eye movements. , 1981, Annual review of neuroscience.

[28]  T. Sejnowski,et al.  A neural model of the cortical representation of egocentric distance. , 1994, Cerebral cortex.

[29]  Alexander Borst,et al.  Principles of visual motion detection , 1989, Trends in Neurosciences.

[30]  J. V. Gisbergen,et al.  Collicular ensemble coding of saccades based on vector summation , 1987, Neuroscience.

[31]  Frans A. J. Verstraten,et al.  The motion aftereffect , 1998, Trends in Cognitive Sciences.

[32]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[33]  Holk Cruse,et al.  Neural networks as cybernetic systems , 1996 .

[34]  Constance S. Royden,et al.  Mathematical analysis of motion-opponent mechanisms used in the determination of heading and depth. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[35]  A. Opstal,et al.  Monkey Superior Colliculus Activity During Short-Term Saccadic Adaptation , 1997, Brain Research Bulletin.

[36]  Peter Thier,et al.  The role of cortical area MST in a model of combined smooth eye-head pursuit , 1999, Biological Cybernetics.

[37]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[38]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[39]  Paul B. Johnson,et al.  Making arm movements within different parts of space: dynamic aspects in the primate motor cortex , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  J. Kalaska,et al.  A comparison of movement direction-related versus load direction- related activity in primate motor cortex, using a two-dimensional reaching task , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  Richard A. Andersen,et al.  Coordinate transformations in the representation of spatial information , 1993, Current Opinion in Neurobiology.

[42]  I. Ohzawa,et al.  Neural mechanisms for encoding binocular disparity: receptive field position versus phase. , 1999, Journal of neurophysiology.

[43]  J A Perrone,et al.  Model for the computation of self-motion in biological systems. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[44]  David J. Fleet,et al.  Neural encoding of binocular disparity: Energy models, position shifts and phase shifts , 1996, Vision Research.

[45]  J H Rieger,et al.  Processing differential image motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[46]  Paul B. Johnson,et al.  Visuomotor transformations underlying arm movements toward visual targets: a neural network model of cerebral cortical operations , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  D Marr,et al.  Directional selectivity and its use in early visual processing , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[48]  H. Barlow,et al.  The mechanism of directionally selective units in rabbit's retina. , 1965, The Journal of physiology.

[49]  J. Tresilian Visually timed action: time-out for ‘tau’? , 1999, Trends in Cognitive Sciences.

[50]  A. V. van den Berg,et al.  Why two eyes are better than one for judgements of heading , 1994, Nature.

[51]  F. Ottes,et al.  Metrics of saccade responses to visual double stimuli: Two different modes , 1984, Vision Research.

[52]  C Koch,et al.  Computing motion in the primate's visual system. , 1989, The Journal of experimental biology.

[53]  Bart Krekelberg,et al.  Postsaccadic visual references generate presaccadic compression of space , 2000, Nature.

[54]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[55]  S. Palmer Vision Science : Photons to Phenomenology , 1999 .

[56]  J. Koenderink,et al.  Second-order optic flow , 1992 .

[57]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[58]  K. Hoffmann,et al.  Optokinetic eye movements elicited by radial optic flow in the macaque monkey. , 1998, Journal of neurophysiology.

[59]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[60]  Gully A. P. C. Burns,et al.  The Analysis of Cortical Connectivity , 1995 .

[61]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[62]  Markus Lappe,et al.  A Neural Network for the Processing of Optic Flow from Ego-Motion in Man and Higher Mammals , 1993, Neural Computation.

[63]  P. Goldman-Rakic,et al.  Preface: Cerebral Cortex Has Come of Age , 1991 .

[64]  M. Lappe,et al.  Interaction of stereo vision and optic flow processing revealed by an illusory stimulus , 1998, Vision Research.

[65]  D J Hannon,et al.  Eye movements and optical flow. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[66]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[67]  F. Bremmer,et al.  The use of optical velocities for distance discrimination and reproduction during visually simulated self motion , 1999, Experimental Brain Research.

[68]  C. Duffy,et al.  Optic flow illusion and single neuron behaviour reconciled by a population model , 1999, The European journal of neuroscience.

[69]  K. Hoffmann,et al.  Eye position effects in monkey cortex. II. Pursuit- and fixation-related activity in posterior parietal areas LIP and 7A. , 1997, Journal of neurophysiology.

[70]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[71]  R. Andersen,et al.  Mechanisms of Heading Perception in Primate Visual Cortex , 1996, Science.

[72]  D. Munoz,et al.  Gaze control in the cat: studies and modeling of the coupling between orienting eye and head movements in different behavioral tasks. , 1990, Journal of neurophysiology.

[73]  H. Honda Perceptual localization of visual stimuli flashed during saccades , 1989, Perception & psychophysics.

[74]  K. Hoffmann,et al.  Optic Flow Processing in Monkey STS: A Theoretical and Experimental Approach , 1996, The Journal of Neuroscience.

[75]  S Ullman,et al.  Maximizing Rigidity: The Incremental Recovery of 3-D Structure from Rigid and Nonrigid Motion , 1984, Perception.

[76]  Bremmer,et al.  Eye position encoding in the macaque posterior parietal cortex , 1998, The European journal of neuroscience.

[77]  E C Hildreth,et al.  Incremental rigidity scheme for recovering structure from motion: position-based versus velocity-based formulations. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[78]  A. Georgopoulos Current issues in directional motor control , 1995, Trends in Neurosciences.

[79]  R. Andersen,et al.  Electrical microstimulation distinguishes distinct saccade-related areas in the posterior parietal cortex. , 1998, Journal of neurophysiology.

[80]  L E Mays,et al.  Neurons in monkey parietal area LIP are tuned for eye-movement parameters in three-dimensional space. , 1995, Journal of neurophysiology.

[81]  D. Sparks,et al.  Population coding of saccadic eye movements by neurons in the superior colliculus , 1988, Nature.

[82]  T. Sejnowski,et al.  Egocentric Spaw Representation in Early Vision , 1993, Journal of Cognitive Neuroscience.

[83]  Thomas Rosemeier,et al.  Interaction of vestibular, somatosensory and visual signals for postural control and motion perception under terrestrial and microgravity conditions—a conceptual model , 1998, Brain Research Reviews.

[84]  J. van Santen,et al.  Elaborated Reichardt detectors. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[85]  J. Daugman Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[86]  Peter Ford Dominey,et al.  A cortico-subcortical model for generation of spatially accurate sequential saccades. , 1992, Cerebral cortex.

[87]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[88]  A. Opstal,et al.  Human eye-head coordination in two dimensions under different sensorimotor conditions , 1997, Experimental Brain Research.

[89]  Markus Lappe,et al.  A model of the combination of optic flow and extraretinal eye movement signals in primate extrastriate visual cortex: Neural model of self-motion from optic flow and extraretinal cues , 1998, Neural Networks.

[90]  G. Orban,et al.  Shape and Spatial Distribution of Receptive Fields and Antagonistic Motion Surrounds in the Middle Temporal Area (V5) of the Macaque , 1995, The European journal of neuroscience.

[91]  Florentin Wörgötter,et al.  How to “hear” visual disparities: real-time stereoscopic spatial depth analysis using temporal resonance , 1998, Biological Cybernetics.

[92]  James E. Cutting,et al.  HIGH-PERFORMANCE COMPUTING AND HUMAN VISION I , 2002 .

[93]  M. Gazzaniga,et al.  The new cognitive neurosciences , 2000 .

[94]  A. V. van den Berg,et al.  Heading detection using motion templates and eye velocity gain fields , 1998, Vision Research.

[95]  C. Galletti,et al.  Gaze-dependent visual neurons in area V3A of monkey prestriate cortex , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[96]  H. Sakata,et al.  Functional properties of visual tracking neurons in posterior parietal association cortex of the monkey. , 1983, Journal of neurophysiology.

[97]  Markus Lappe,et al.  How stereovision interacts with optic flow perception: neural mechanisms , 1999, Neural Networks.

[98]  Ian P. Howard,et al.  Binocular Vision and Stereopsis , 1996 .

[99]  Masao Ito Cerebellar learning in the vestibulo–ocular reflex , 1998, Trends in Cognitive Sciences.

[100]  R. Wurtz,et al.  Pursuit and optokinetic deficits following chemical lesions of cortical areas MT and MST. , 1988, Journal of neurophysiology.

[101]  M. Goldberg,et al.  Spatial processing in the monkey frontal eye field. I. Predictive visual responses. , 1997, Journal of neurophysiology.

[102]  Hermann Wagner,et al.  Disparity-sensitive cells in the owl have a characteristic disparity , 1993, Nature.

[103]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[104]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[105]  K. Hepp,et al.  Theoretical explanations of Listing's law and their implication for binocular vision , 1995, Vision Research.

[106]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[107]  T. Sejnowski,et al.  Learning and memory in the vestibulo-ocular reflex. , 1995, Annual review of neuroscience.

[108]  L. Optican,et al.  Model of the control of saccades by superior colliculus and cerebellum. , 1999, Journal of neurophysiology.

[109]  L. Matin,et al.  Visual Perception of Direction for Stimuli Flashed During Voluntary Saccadic Eye Movements , 1965, Science.

[110]  Christian Quaia,et al.  The maintenance of spatial accuracy by the perisaccadic remapping of visual receptive fields , 1998, Neural Networks.

[111]  H. Wilson,et al.  A psychophysically motivated model for two-dimensional motion perception , 1992, Visual Neuroscience.

[112]  Michael I. Jordan,et al.  A more biologically plausible learning rule for neural networks. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[113]  R. Andersen,et al.  A Computational Framework for Determining Stereo Correspondence from a Set of Linear Spatial Filters Perception of Three-dimensional Structure from Motion Review , 2022 .

[114]  M. Goodale,et al.  The visual brain in action , 1995 .

[115]  F A Miles,et al.  The neural processing of 3‐D visual information: evidence from eye movements , 1998, The European journal of neuroscience.

[116]  W. Berger,et al.  Visual influence on human locomotion Modulation to changes in optic flow , 1997, Experimental Brain Research.

[117]  K. Hoffmann,et al.  Linear Vestibular Self‐Motion Signals in Monkey Medial Superior Temporal Area , 1999, Annals of the New York Academy of Sciences.

[118]  Jonathan S. Turner,et al.  Towards a framework for high speed communication in a heterogeneous networking environment , 1990, IEEE INFOCOM '89, Proceedings of the Eighth Annual Joint Conference of the IEEE Computer and Communications Societies.

[119]  Lee Dn,et al.  The optic flow field: the foundation of vision. , 1980 .

[120]  T. Albright,et al.  Contribution of area MT to perception of three-dimensional shape: a computational study , 1996, Vision Research.

[121]  K. Kopecz,et al.  Saccadic reaction times in gap/overlap paradigms: a model based on integration of intentional and visual information on neural, dynamic fields , 1995, Vision Research.

[122]  Markus Lappe,et al.  Functional Consequences of an Integration of Motion and Stereopsis in Area MT of Monkey Extrastriate Visual Cortex , 1996, Neural Computation.

[123]  Ning Qian,et al.  Binocular Receptive Field Models, Disparity Tuning, and Characteristic Disparity , 1996, Neural Computation.

[124]  C. Scudder A new local feedback model of the saccadic burst generator. , 1988, Journal of neurophysiology.

[125]  R. Wurtz,et al.  Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. , 1995, Journal of neurophysiology.

[126]  Ning Qian,et al.  Relationship Between Phase and Energy Methods for Disparity Computation , 2000, Neural Computation.

[127]  F. Bremmer,et al.  Perception of self-motion from visual flow , 1999, Trends in Cognitive Sciences.

[128]  P. R. Snoeren,et al.  EXTRACTION OF THREE-DIMENSIONAL SHAPE FROM OPTIC FLOW : A GEOMETRIC APPROACH , 1994 .

[129]  Maninder K. Kahlon,et al.  Visual Motion Analysis for Pursuit Eye Movements in Area MT of Macaque Monkeys , 1999, The Journal of Neuroscience.

[130]  D. Burr,et al.  Selective suppression of the magnocellular visual pathway during saccadic eye movements , 1994, Nature.

[131]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[132]  H. Komatsu,et al.  Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. , 1988, Journal of neurophysiology.

[133]  J F Soechting,et al.  Moving in three-dimensional space: frames of reference, vectors, and coordinate systems. , 1992, Annual review of neuroscience.

[134]  Bruce Bridgeman,et al.  Failure to detect displacement of the visual world during saccadic eye movements , 1975, Vision Research.

[135]  K. Hoffmann,et al.  Ocular responses to radial optic flow and single accelerated targets in humans , 1999, Vision Research.

[136]  Norberto M. Grzywacz,et al.  A computational theory for the perception of coherent visual motion , 1988, Nature.

[137]  M. Lappe Neuronal processing of optic flow , 2000 .

[138]  P. McOwan,et al.  A computational model of the analysis of some first-order and second-order motion patterns by simple and complex cells , 1992, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[139]  F A Miles,et al.  Ocular responses to translation and their dependence on viewing distance. II. Motion of the scene. , 1991, Journal of neurophysiology.

[140]  Gian F. Poggio Mechanisms of Stereopsis in Monkey Visual Cortex , 1995 .

[141]  J. Perrone,et al.  A model of self-motion estimation within primate extrastriate visual cortex , 1994, Vision Research.

[142]  M. Goldberg,et al.  Neurons in the monkey superior colliculus predict the visual result of impending saccadic eye movements. , 1995, Journal of neurophysiology.

[143]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[144]  P. Thier,et al.  Eye movements of rhesus monkeys directed towards imaginary targets , 1999, Vision Research.

[145]  Michele A. Basso,et al.  Modulation of neuronal activity by target uncertainty , 1997, Nature.

[146]  R. Andersen,et al.  Encoding of three-dimensional structure-from-motion by primate area MT neurons , 1998, Nature.

[147]  C. Galletti,et al.  Eye Position Influence on the Parieto‐occipital Area PO (V6) of the Macaque Monkey , 1995, The European journal of neuroscience.

[148]  K. Hoffmann,et al.  Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. , 1997, Journal of neurophysiology.

[149]  J. Koenderink,et al.  Extraction of optical velocity by use of multi-input Reichardt detectors. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[150]  S. Lehky,et al.  Neural model of stereoacuity and depth interpolation based on a distributed representation of stereo disparity [published erratum appears in J Neurosci 1991 Mar;11(3):following Table of Contents] , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[151]  S. Scott,et al.  Changes in motor cortex activity during reaching movements with similar hand paths but different arm postures. , 1995, Journal of neurophysiology.

[152]  D. Heeger Modeling simple-cell direction selectivity with normalized, half-squared, linear operators. , 1993, Journal of neurophysiology.

[153]  K. Hoffmann,et al.  Motion perception during saccades , 1993, Vision Research.

[154]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[155]  T. Vilis,et al.  Geometric relations of eye position and velocity vectors during saccades , 1990, Vision Research.

[156]  E. J. Vrijenhoek,et al.  Arm position constraints during pointing and reaching in 3-D space. , 1997, Journal of neurophysiology.

[157]  L. Abbott,et al.  A model of multiplicative neural responses in parietal cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.