Multi-Sensory Weights Depend on Contextual Noise in Reference Frame Transformations

During reach planning, we integrate multiple senses to estimate the location of the hand and the target, which is used to generate a movement. Visual and proprioceptive information are combined to determine the location of the hand. The goal of this study was to investigate whether multi-sensory integration is affected by extraretinal signals, such as head roll. It is believed that a coordinate matching transformation is required before vision and proprioception can be combined because proprioceptive and visual sensory reference frames do not generally align. This transformation utilizes extraretinal signals about current head roll position, i.e., to rotate proprioceptive signals into visual coordinates. Since head roll is an estimated sensory signal with noise, this head roll dependency of the reference frame transformation should introduce additional noise to the transformed signal, reducing its reliability and thus its weight in the multi-sensory integration. To investigate the role of noisy reference frame transformations on multi-sensory weighting, we developed a novel probabilistic (Bayesian) multi-sensory integration model (based on Sober and Sabes, 2003) that included explicit (noisy) reference frame transformations. We then performed a reaching experiment to test the model's predictions. To test for head roll dependent multi-sensory integration, we introduced conflicts between viewed and actual hand position and measured reach errors. Reach analysis revealed that eccentric head roll orientations led to an increase of movement variability, consistent with our model. We further found that the weighting of vision and proprioception depended on head roll, which we interpret as being a result of signal dependant noise. Thus, the brain has online knowledge of the statistics of its internal sensory representations. In summary, we show that sensory reliability is used in a context-dependent way to adjust multi-sensory integration weights for reaching.

[1]  Satoshi Hirose,et al.  Activity in posterior parietal cortex mediates the visual dominance over kinesthesia , 2007, Neuroscience Research.

[2]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[3]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[4]  Steve W. C. Chang,et al.  Using a Compound Gain Field to Compute a Reach Plan , 2009, Neuron.

[5]  Lacquaniti,et al.  Visuo‐motor transformations for arm reaching , 1998, The European journal of neuroscience.

[6]  S. Lechner-Steinleitner,et al.  Interaction of labyrinthine and somatoreceptor inputs as determinants of the subjective vertical , 1978, Psychological research.

[7]  D. McCloskey,et al.  Joint sense, muscle sense, and their combination as position sense, measured at the distal interphalangeal joint of the middle finger. , 1976, The Journal of physiology.

[8]  M S Landy,et al.  Ideal cue combination for localizing texture-defined edges. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  R. J. van Beers,et al.  Integration of proprioceptive and visual position-information: An experimentally supported model. , 1999, Journal of neurophysiology.

[10]  Christopher A. Buneo,et al.  Direct visuomotor transformations for reaching , 2002, Nature.

[11]  Demetri Terzopoulos,et al.  Heads up!: biomechanical modeling and neuromuscular control of the neck , 2006, ACM Trans. Graph..

[12]  M. Mon-Williams,et al.  Synaesthesia in the normal limb , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[13]  D. Burr,et al.  Auditory dominance over vision in the perception of interval duration , 2009, Experimental Brain Research.

[14]  J. V. Van Gisbergen,et al.  Properties of the internal representation of gravity inferred from spatial-direction and body-tilt estimates. , 2000, Journal of neurophysiology.

[15]  C. Prablanc,et al.  Vectorial coding of movement: vision, proprioception, or both? , 1995, Journal of neurophysiology.

[16]  Philip N. Sabes,et al.  Flexible strategies for sensory integration during motor planning , 2005, Nature Neuroscience.

[17]  J. F. Soechting,et al.  Oculocentric frames of reference for limb movement. , 2002, Archives italiennes de biologie.

[18]  Michael I. Jordan,et al.  A Model of the Learning of Arm Trajectories from Spatial Deviations , 1994, Journal of Cognitive Neuroscience.

[19]  Philip N. Sabes,et al.  Multisensory Integration during Motor Planning , 2003, The Journal of Neuroscience.

[20]  G E Loeb,et al.  The computation of position sense from spindles in mono- and multiarticular muscles , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  H. Collewijn,et al.  Human ocular counterroll: assessment of static and dynamic properties from electromagnetic scleral coil recordings , 2004, Experimental Brain Research.

[22]  Michael I. Jordan,et al.  Computational models of sensorimotor integration , 1997 .

[23]  D. Burke,et al.  The responses of human muscle spindle endings to vibration during isometric contraction. , 1976, The Journal of physiology.

[24]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[25]  L. Pinneo.  On noise in the nervous system. , 1966, Psychological review.

[26]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[27]  A. Pouget,et al.  Efficient computation and cue integration with noisy population codes , 2001, Nature Neuroscience.

[28]  M. Chacron,et al.  Neural Variability, Detection Thresholds, and Information Transmission in the Vestibular System , 2007, Journal of Neuroscience.

[29]  J. D. Crawford,et al.  Comparing limb proprioception and oculomotor signals during hand-guided saccades , 2007, Experimental Brain Research.

[30]  Gunnar Blohm,et al.  Decoding the cortical transformations for visually guided reaching in 3D space. , 2009, Cerebral cortex.

[31]  V. Henn,et al.  Static roll and pitch in the monkey: Shift and rotation of listing's plane , 1992, Vision Research.

[32]  W Li,et al.  Visual Direction Is Corrected by a Hybrid Extraretinal Eye Position Signal a , 1992, Annals of the New York Academy of Sciences.

[33]  F. Lacquaniti,et al.  Viewer-centered frame of reference for pointing to memorized targets in three-dimensional space. , 1997, Journal of neurophysiology.

[34]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[35]  Gunnar Blohm,et al.  Computations for geometrically accurate visually guided reaching in 3-D space. , 2007, Journal of vision.

[36]  W. Abend,et al.  Response to static tilts of peripheral neurons innervating otolith organs of the squirrel monkey. , 1972, Journal of neurophysiology.

[37]  R. Andersen,et al.  The posterior parietal cortex: Sensorimotor interface for the planning and online control of visually guided movements , 2006, Neuropsychologia.

[38]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[39]  Joshua B. Tenenbaum,et al.  Causal inference in sensorimotor integration , 2006, NIPS.

[40]  L. Harris,et al.  The subjective visual vertical and the perceptual upright , 2006, Experimental Brain Research.

[41]  Denise Taylor,et al.  Head and Neck Position Sense , 2008, Sports medicine.

[42]  D. Angelaki,et al.  Multisensory integration: resolving sensory ambiguities to build novel representations , 2010, Current Opinion in Neurobiology.

[43]  Yale E. Cohen,et al.  A common reference frame for movement plans in the posterior parietal cortex , 2002, Nature Reviews Neuroscience.

[44]  I. Curthoys,et al.  The Effect of Ocular Torsional Position on Perception of the Roll-tilt of Visual Stimuli , 1997, Vision Research.

[45]  W. A. Fletcher,et al.  Eye position signals in human saccadic processing , 2004, Experimental Brain Research.

[46]  R. Jacobs,et al.  Experience-dependent visual cue integration based on consistencies between visual and haptic percepts , 2001, Vision Research.

[47]  D. Wolpert,et al.  When Feeling Is More Important Than Seeing in Sensorimotor Adaptation , 2002, Current Biology.

[48]  R. Andersen,et al.  The influence of the angle of gaze upon the excitability of the light- sensitive neurons of the posterior parietal cortex , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[49]  D. McCloskey,et al.  The contribution of muscle afferents to kinaesthesia shown by vibration induced illusions of movement and by the effects of paralysing joint afferents. , 1972, Brain : a journal of neurology.

[50]  F. J. Clark,et al.  Slowly adapting receptors in cat knee joint: can they signal joint angle? , 1975, Journal of neurophysiology.

[51]  J D Crawford,et al.  Proprioceptive guidance of saccades in eye-hand coordination. , 2006, Journal of neurophysiology.

[52]  B. Edin,et al.  Muscle afferent responses to isometric contractions and relaxations in humans. , 1990, Journal of neurophysiology.

[53]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[54]  Sabine M P Verschueren,et al.  Position sensitivity of human muscle spindles: single afferent and population representations. , 2002, Journal of neurophysiology.

[55]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[56]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[57]  Philip N. Sabes,et al.  Sensory transformations and the use of multiple reference frames for reach planning , 2009, Nature Neuroscience.