Visual cues, expectations, and sensorimotor memories in the prediction and perception of object dynamics during manipulation

When we grasp and lift novel objects, we rely on visual cues and sensorimotor memories to predictively scale our finger forces and exert compensatory torques according to object properties. Recently, it was shown that object appearance, previous force scaling errors, and previous torque compensation errors strongly impact our percept. However, the influence of visual geometric cues on the perception of object torques and weights in a grasp to lift task is poorly understood. Moreover, little is known about how visual cues, prior expectations, sensory feedback, and sensorimotor memories are integrated for anticipatory torque control and object perception. Here, 12 young and 12 elderly participants repeatedly grasped and lifted an object while trying to prevent object tilt. Before each trial, we randomly repositioned both the object handle, providing a geometric cue on the upcoming torque, as well as a hidden weight, adding an unforeseeable torque variation. Before lifting, subjects indicated their torque expectations, as well as reporting their experience of torque and weight after each lift. Mixed-effect multiple regression models showed that visual shape cues governed anticipatory torque compensation, whereas sensorimotor memories played less of a role. In contrast, the external torque and committed compensation errors at lift-off mainly determined how object torques and weight were perceived. The modest effect of handle position differed for torque and weight perception. Explicit torque expectations were also correlated with anticipatory torque compensation and torque perception. Our main findings generalized across both age groups. Our results suggest distinct weighting of inputs for action and perception according to reliability.

[1]  B. Richardson,et al.  Visual and haptic influence on perception of stimulus size , 2010, Attention, perception & psychophysics.

[2]  K. J. Cole,et al.  Sensorimotor Memory For Fingertip Forces: Evidence For A Task-Independent Motor Memory , 2003, The Journal of Neuroscience.

[3]  M. Ernst,et al.  Optimal integration of shape information from vision and touch , 2007, Experimental Brain Research.

[4]  K. J. Cole,et al.  Old age impairs the use of arbitrary visual cues for predictive control of fingertip forces during grasp , 2002, Experimental Brain Research.

[5]  J. Flanagan,et al.  Effects of surface texture on weight perception when lifting objects with a precision grip , 1995, Perception & psychophysics.

[6]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[7]  R. Johansson,et al.  Visual size cues in the programming of manipulative forces during precision grip , 2004, Experimental Brain Research.

[8]  Marco Santello,et al.  Role of digit placement control in sensorimotor transformations for dexterous manipulation. , 2017, Journal of neurophysiology.

[9]  David W. Franklin,et al.  Computational Mechanisms of Sensorimotor Control , 2011, Neuron.

[10]  G. Buckingham Getting a grip on heaviness perception: a review of weight illusions and their probable causes , 2014, Experimental Brain Research.

[11]  R. Johansson,et al.  Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with precision grip , 2004, Experimental Brain Research.

[12]  S. Levinson,et al.  WEIRD languages have misled us, too , 2010, Behavioral and Brain Sciences.

[13]  Marco Santello,et al.  Retention and interference of learned dexterous manipulation: interaction between multiple sensorimotor processes. , 2015, Journal of neurophysiology.

[14]  Melvyn A. Goodale,et al.  Size Matters: A Single Representation Underlies Our Perceptions of Heaviness in the Size-Weight Illusion , 2013, PloS one.

[15]  Louise P. Kirsch,et al.  Information about the Weight of Grasped Objects from Vision and Internal Models Interacts within the Primary Motor Cortex , 2010, The Journal of Neuroscience.

[16]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[17]  K. J. Cole,et al.  Mechanisms for Age-Related Changes of Fingertip Forces during Precision Gripping and Lifting in Adults , 1999, The Journal of Neuroscience.

[18]  Daniel M Wolpert,et al.  Computational principles of sensorimotor control that minimize uncertainty and variability , 2007, The Journal of physiology.

[19]  Matthijs Verhage,et al.  A solution to dependency: using multilevel analysis to accommodate nested data , 2014, Nature Neuroscience.

[20]  S. Lederman,et al.  The role of haptic versus visual volume cues in the size-weight illusion , 1993, Perception & psychophysics.

[21]  C. B. Colby The weirdest people in the world , 1973 .

[22]  D. Wolpert,et al.  When Feeling Is More Important Than Seeing in Sensorimotor Adaptation , 2002, Current Biology.

[23]  Marco Santello,et al.  Context-Dependent Learning Interferes with Visuomotor Transformations for Manipulation Planning , 2012, The Journal of Neuroscience.

[24]  Marco Davare,et al.  Sensorimotor Memory Biases Weight Perception During Object Lifting , 2015, Front. Hum. Neurosci..

[25]  Joachim Hermsdörfer,et al.  Torque-planning errors affect the perception of object properties and sensorimotor memories during object manipulation in uncertain grasp situations. , 2019, Journal of neurophysiology.

[26]  R. J. van Beers,et al.  Integration of proprioceptive and visual position-information: An experimentally supported model. , 1999, Journal of neurophysiology.

[27]  A. Gordon,et al.  Selective use of visual information signaling objects' center of mass for anticipatory control of manipulative fingertip forces , 2003, Experimental Brain Research.

[28]  Jonathan S. Cant,et al.  Living in a material world: how visual cues to material properties affect the way that we lift objects and perceive their weight. , 2009, Journal of neurophysiology.

[29]  J. Henrich,et al.  The weirdest people in the world? , 2010, Behavioral and Brain Sciences.

[30]  M. Santello,et al.  Anticipatory Planning and Control of Grasp Positions and Forces for Dexterous Two-Digit Manipulation , 2010, The Journal of Neuroscience.

[31]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[32]  J. Zwislocki,et al.  Absolute scaling of sensory magnitudes: A validation , 1980, Perception & psychophysics.

[33]  D. Wolpert,et al.  Two Eyes for an Eye: The Neuroscience of Force Escalation , 2003, Science.

[34]  K. J. Cole Grasp force control in older adults. , 1991, Journal of motor behavior.

[35]  Anita Christie,et al.  Systematic review and meta-analysis of skeletal muscle fatigue in old age. , 2011, Medicine and science in sports and exercise.

[36]  Jeffrey D. Long,et al.  Longitudinal Data Analysis for the Behavioral Sciences Using R , 2011 .

[37]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[38]  Monica Gori,et al.  Parkinson's disease accelerates age-related decline in haptic perception by altering somatosensory integration. , 2012, Brain : a journal of neurology.

[39]  Jason Y. Choi,et al.  Grasping uncertainty: effects of sensorimotor memories on high-level planning of dexterous manipulation. , 2013, Journal of neurophysiology.

[40]  J. Randall Flanagan,et al.  The role of observers' gaze behaviour when watching object manipulation tasks: predicting and evaluating the consequences of action , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[41]  Marco Santello,et al.  Manipulation after object rotation reveals independent sensorimotor memory representations of digit positions and forces. , 2010, Journal of neurophysiology.

[42]  J. Flanagan,et al.  Distinct contributions of explicit and implicit memory processes to weight prediction when lifting objects and judging their weights: an aging study. , 2016, Journal of neurophysiology.

[43]  G. Buckingham,et al.  The weight of expectation: Implicit, rather than explicit, prior expectations drive the size–weight illusion , 2016, Quarterly journal of experimental psychology.

[44]  I Salimi,et al.  Specificity of internal representations underlying grasping. , 2000, Journal of neurophysiology.

[45]  Robert Volcic,et al.  Grasping movements toward seen and handheld objects , 2019, Scientific Reports.

[46]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[47]  D. Wolpert,et al.  Ageing increases reliance on sensorimotor prediction through structural and functional differences in frontostriatal circuits , 2016, Nature Communications.

[48]  D. Bates,et al.  Parsimonious Mixed Models , 2015, 1506.04967.

[49]  J R Flanagan,et al.  Coming to grips with weight perception: Effects of grasp configuration on perceived heaviness , 2000, Perception & psychophysics.

[50]  D. Wolpert,et al.  Attenuation of Self-Generated Tactile Sensations Is Predictive, not Postdictive , 2006, PLoS biology.

[51]  J. Flanagan,et al.  Independence of perceptual and sensorimotor predictions in the size–weight illusion , 2000, Nature Neuroscience.

[52]  K. J. Cole Lifting a familiar object: visual size analysis, not memory for object weight, scales lift force , 2008, Experimental Brain Research.

[53]  R. Johansson,et al.  Experience Can Change Distinct Size-Weight Priors Engaged in Lifting Objects and Judging their Weights , 2008, Current Biology.

[54]  Marco Santello,et al.  Effects of Visual Cues of Object Density on Perception and Anticipatory Control of Dexterous Manipulation , 2013, PloS one.

[55]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[56]  R. Johansson,et al.  Action plans used in action observation , 2003, Nature.

[57]  Satoru Suzuki,et al.  Visual Influence on Haptic Torque Perception , 2012, Perception.

[58]  Peter R. Francis,et al.  A comparison of prehension force control in young and elderly individuals , 1996, European Journal of Applied Physiology and Occupational Physiology.

[59]  Marco Santello,et al.  Visual Cues of Object Properties Differentially Affect Anticipatory Planning of Digit Forces and Placement , 2016, PloS one.

[60]  Joachim Hermsdörfer,et al.  Anticipatory scaling of grip forces when lifting objects of everyday life , 2011, Experimental Brain Research.