The Role of Haptic Expectations in Reaching to Grasp: From Pantomime to Natural Grasps and Back Again

When we reach to pick up an object, our actions are effortlessly informed by the object’s spatial information, the position of our limbs, stored knowledge of the object’s material properties, and what we want to do with the object. A substantial body of evidence suggests that grasps are under the control of “automatic, unconscious” sensorimotor modules housed in the “dorsal stream” of the posterior parietal cortex. Visual online feedback has a strong effect on the hand’s in-flight grasp aperture. Previous work of ours exploited this effect to show that grasps are refractory to cued expectations for visual feedback. Nonetheless, when we reach out to pretend to grasp an object (pantomime grasp), our actions are performed with greater cognitive effort and they engage structures outside of the dorsal stream, including the ventral stream. Here we ask whether our previous finding would extend to cued expectations for haptic feedback. Our method involved a mirror apparatus that allowed participants to see a “virtual” target cylinder as a reflection in the mirror at the start of all trials. On “haptic feedback” trials, participants reached behind the mirror to grasp a size-matched cylinder, spatially coincident with the virtual one. On “no-haptic feedback” trials, participants reached behind the mirror and grasped into “thin air” because no cylinder was present. To manipulate haptic expectation, we organized the haptic conditions into blocked, alternating, and randomized schedules with and without verbal cues about the availability of haptic feedback. Replicating earlier work, we found the strongest haptic effects with the blocked schedules and the weakest effects in the randomized uncued schedule. Crucially, the haptic effects in the cued randomized schedule was intermediate. An analysis of the influence of the upcoming and immediately preceding haptic feedback condition in the cued and uncued random schedules showed that cuing the upcoming haptic condition shifted the haptic influence on grip aperture from the immediately preceding trial to the upcoming trial. These findings indicate that, unlike cues to the availability of visual feedback, participants take advantage of cues to the availability of haptic feedback, flexibly engaging pantomime, and natural modes of grasping to optimize the movement.

[1]  J. Marotta,et al.  The influence of the Sander parallelogram illusion and early, middle and late vision on goal-directed reaching and grasping , 2020, Experimental Brain Research.

[2]  T. Ganel,et al.  Consciously monitored grasping is vulnerable to perceptual intrusions , 2020, Consciousness and Cognition.

[3]  Robert L. Whitwell,et al.  Grip Constancy but Not Perceptual Size Constancy Survives Lesions of Early Visual Cortex , 2020, Current Biology.

[4]  Robert L. Whitwell,et al.  Grip Constancy but Not Perceptual Size Constancy Survives Lesions of Early Visual Cortex , 2020, Current Biology.

[5]  C. Hesse,et al.  Grasping and perception are both affected by irrelevant information and secondary tasks: new evidence from the Garner paradigm , 2019, Psychological Research.

[6]  C. Becchio,et al.  A kind of magic: Enhanced detection of pantomimed grasps in professional magicians , 2020, Quarterly Journal of Experimental Psychology.

[7]  Marco Santello,et al.  Dexterous Object Manipulation Requires Context-Dependent Sensorimotor Cortical Interactions in Humans , 2019, Cerebral cortex.

[8]  E. Brenner,et al.  Why some size illusions affect grip aperture , 2019, Experimental Brain Research.

[9]  T. Isa.  Dexterous Hand Movements and Their Recovery After Central Nervous System Injury. , 2019, Annual review of neuroscience.

[10]  T. Ganel,et al.  Weber’s law in 2D and 3D grasping , 2019, Psychological research.

[11]  Agnieszka M. Nowik,et al.  The neural underpinnings of haptically guided functional grasping of tools: An fMRI study , 2019, NeuroImage.

[12]  Robert L. Whitwell,et al.  Visuomotor adaptation in the absence of input from early visual cortex , 2019, Cortex.

[13]  I. Camponogara,et al.  Grasping movements toward seen and handheld objects , 2019, Scientific Reports.

[14]  B. Tversky,et al.  The kinematics that you do not expect: Integrating prior information and kinematics to understand intentions , 2019, Cognition.

[15]  F. Domini,et al.  The endless visuomotor calibration of reach-to-grasp actions , 2018, Scientific Reports.

[16]  Jacqueline C. Snow,et al.  What Role Does “Elongation” Play in “Tool-Specific” Activation and Connectivity in the Dorsal and Ventral Visual Streams? , 2018, Cerebral cortex.

[17]  Fulvio Domini,et al.  How removing visual information affects grasping movements , 2018, Experimental Brain Research.

[18]  T. Ganel,et al.  Dissociable effects of irrelevant context on 2D and 3D grasping , 2018, Attention, perception & psychophysics.

[19]  M. Goodale,et al.  The Sander parallelogram illusion dissociates action and perception despite control for the litany of past confounds , 2018, Cortex.

[20]  R. Cañal-Bruland,et al.  Nothing magical: pantomimed grasping is controlled by the ventral system , 2017, Experimental Brain Research.

[21]  G. Króliczak,et al.  Planning Functional Grasps of Simple Tools Invokes the Hand-independent Praxis Representation Network: An fMRI Study , 2017, Journal of the International Neuropsychological Society.

[22]  Philippe A. Chouinard,et al.  Rapid decrement in the effects of the Ponzo display dissociates action and perception , 2016, Psychonomic bulletin & review.

[23]  Volker H. Franz,et al.  The functional subdivision of the visual brain: Is there a real illusion effect on action? A multi-lab replication study , 2016, Cortex.

[24]  Shirin Davarpanah Jazi,et al.  Pantomime-Grasping: Advance Knowledge of Haptic Feedback Availability Supports an Absolute Visuo-Haptic Calibration , 2016, Front. Hum. Neurosci..

[25]  Jody C. Culham,et al.  Do human brain areas involved in visuomotor actions show a preference for real tools over visually similar non-tools? , 2015, Neuropsychologia.

[26]  V. Franz,et al.  Garner-Interference in left-handed awkward grasping , 2014, Psychological Research.

[27]  Melvyn A. Goodale,et al.  Real-time vision, tactile cues, and visual form agnosia: removing haptic feedback from a “natural” grasping task induces pantomime-like grasps , 2015, Front. Hum. Neurosci..

[28]  Melvyn A. Goodale,et al.  Patient DF’s visual brain in action: Visual feedforward control in visual form agnosia , 2015, Vision Research.

[29]  V. Franz,et al.  Semantic grasping escapes Weber's law , 2015, Neuropsychologia.

[30]  Erez Freud,et al.  Visual control of action directed toward two-dimensional objects relies on holistic processing of object shape , 2015, Psychonomic bulletin & review.

[31]  Afra Wohlschläger,et al.  The Neural Correlates of Planning and Executing Actual Tool Use , 2014, The Journal of Neuroscience.

[32]  J. Randall Flanagan,et al.  Representation of Object Weight in Human Ventral Visual Cortex , 2014, Current Biology.

[33]  Melvyn A. Goodale,et al.  DF's visual brain in action: The role of tactile cues , 2014, Neuropsychologia.

[34]  M. Goodale,et al.  Variability-based Garner interference for perceptual estimations but not for grasping , 2014, Experimental Brain Research.

[35]  G. Humphreys,et al.  A Case Study in Visual Agnosia Revisited: To see but not to see , 2013 .

[36]  A. McMahan.  Immersion, Engagement, and Presence: A Method for Analyzing 3-D Video Games , 2013 .

[37]  M. Heath,et al.  Distinct Visual Cues Mediate Aperture Shaping for Grasping and Pantomime-Grasping Tasks , 2013, Journal of motor behavior.

[38]  T. Inui,et al.  How Vision Affects Kinematic Properties of Pantomimed Prehension Movements , 2013, Front. Psychology.

[39]  V. Franz,et al.  Inferences about time course of Weber’s Law violate statistical principles , 2013, Vision Research.

[40]  Matthew Heath,et al.  Grasping time does not influence the early adherence of aperture shaping to Weber's law , 2012, Front. Hum. Neurosci..

[41]  J. Hermsdörfer,et al.  Tool use without a tool: kinematic characteristics of pantomiming as compared to actual use and the effect of brain damage , 2012, Experimental Brain Research.

[42]  Thomas Schenk,et al.  No Dissociation between Perception and Action in Patient DF When Haptic Feedback is Withdrawn , 2012, The Journal of Neuroscience.

[43]  M. Heath,et al.  Visually and memory-guided grasping: Aperture shaping exhibits a time-dependent scaling to Weber’s law , 2011, Vision Research.

[44]  Melvyn A. Goodale,et al.  Grasping the non-conscious: Preserved grip scaling to unseen objects for immediate but not delayed grasping following a unilateral lesion to primary visual cortex , 2011, Vision Research.

[45]  Dwight J. Kravitz,et al.  A new neural framework for visuospatial processing , 2011, Nature Reviews Neuroscience.

[46]  A. Milner,et al.  The Magic Grasp: Motor Expertise in Deception , 2011, PloS one.

[47]  C. Hesse,et al.  Grasping remembered objects: Exponential decay of the visual memory , 2010, Vision Research.

[48]  R. Newport,et al.  The continuous updating of grasp in response to dynamic changes in object size, hand size and distractor proximity , 2010, Neuropsychologia.

[49]  G. R. Fink,et al.  The temporal dynamics of the Müller-Lyer illusion. , 2010, Cerebral cortex.

[50]  Johannes Rüter,et al.  The Anatomy of Object Recognition—Visual Form Agnosia Caused by Medial Occipitotemporal Stroke , 2009, The Journal of Neuroscience.

[51]  Melvyn A. Goodale,et al.  Updating the programming of a precision grip is a function of recent history of available feedback , 2009, Experimental Brain Research.

[52]  Tzvi Ganel,et al.  Visual coding for action violates fundamental psychophysical principles , 2008, Current Biology.

[53]  Benoni B. Edin,et al.  Prediction of object contact during grasping , 2008, Experimental Brain Research.

[54]  S. Frey.  Tool use, communicative gesture and cerebral asymmetries in the modern human brain , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[55]  Geoffrey P. Bingham,et al.  Calibrating grasp size and reach distance: interactions reveal integral organization of reaching-to-grasp movements , 2008, Experimental Brain Research.

[56]  Melvyn A. Goodale,et al.  Grasping future events: explicit knowledge of the availability of visual feedback fails to reliably influence prehension , 2008, Experimental Brain Research.

[57]  Mark Mon-Williams,et al.  Natural prehension in trials without haptic feedback but only when calibration is allowed , 2007, Neuropsychologia.

[58]  J. Hermsdörfer,et al.  Neural representations of pantomimed and actual tool use: Evidence from an event-related fMRI study , 2007, NeuroImage.

[59]  S. Frey.  What Puts the How in Where? Tool Use and the Divided Visual Streams Hypothesis , 2007, Cortex.

[60]  Emily S. Cross,et al.  On-line grasp control is mediated by the contralateral hemisphere , 2007, Brain Research.

[61]  O. Bock,et al.  Adaptation of grasping responses to distorted object size and orientation , 2007, Experimental Brain Research.

[62]  M. Jeannerod,et al.  Visual cognition: a new look at the two-visual systems model , 2005, Neuropsychologia.

[63]  James Danckert,et al.  Blindsight in action: what can the different sub-types of blindsight tell us about the control of visually guided actions? , 2005, Neuroscience & Biobehavioral Reviews.

[64]  Scott T. Grafton,et al.  Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp , 2005, Nature Neuroscience.

[65]  J Hermsdörfer,et al.  The effect of tactile feedback on pantomime of tool use in apraxia , 2004, Neurology.

[66]  B. Edin,et al.  Task requirements influence sensory integration during grasping in humans. , 2004, Learning & memory.

[67]  M. Goodale,et al.  Visual control of action but not perception requires analytical processing of object shape , 2003, Nature.

[68]  M. Goodale,et al.  Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. , 2003, Brain : a journal of neurology.

[69]  G. Rizzolatti,et al.  Two different streams form the dorsal visual system: anatomy and functions , 2003, Experimental Brain Research.

[70]  Melvyn A. Goodale,et al.  Grasping two-dimensional images and three-dimensional objects in visual-form agnosia , 2002, Experimental Brain Research.

[71]  D. Rosenbaum,et al.  Posture-based motion planning: applications to grasping. , 2001, Psychological review.

[72]  Sarah H. Creem,et al.  Grasping objects by their handles: a necessary interaction between cognition and action. , 2001, Journal of experimental psychology. Human perception and performance.

[73]  L. Buxbaum,et al.  The Role of the Dynamic Body Schema in Praxis: Evidence from Primary Progressive Apraxia , 2000, Brain and Cognition.

[74]  David A. Westwood,et al.  Pantomimed actions may be controlled by the ventral visual stream , 2000, Experimental Brain Research.

[75]  E. Brenner,et al.  A new view on grasping. , 1999, Motor control.

[76]  A. Milner,et al.  The perception and prehension of objects oriented in the depth plane II. Dissociated orientation functions in normal subjects , 1998, Experimental Brain Research.

[77]  U. Castiello.  Grasping a fruit: selection for action. , 1996, Journal of experimental psychology. Human perception and performance.

[78]  A. Milner,et al.  Visuomotor sensitivity for shape and orientation in a patient with visual form agnosia , 1996, Neuropsychologia.

[79]  Y Rossetti,et al.  Grasping without form discrimination in a hemianopic field , 1996, Neuroreport.

[80]  L. Jakobson,et al.  Differences in the visual control of pantomimed and natural grasping movements , 1994, Neuropsychologia.

[81]  H. Bülthoff,et al.  Separate neural pathways for the visual analysis of object shape in perception and prehension , 1994, Current Biology.

[82]  M. Jeannerod,et al.  Impairment of grasping movements following a bilateral posterior parietal lesion , 1994, Neuropsychologia.

[83]  L. Jakobson,et al.  A kinematic analysis of reaching and grasping movements in a patient recovering from optic ataxia , 1991, Neuropsychologia.

[84]  L. Jakobson,et al.  A neurological dissociation between perceiving objects and grasping them , 1991, Nature.

[85]  Robert L. Whitwell,et al.  Biomechanical constraints do not influence pantomime-grasping adherence to Weber’s law: A reply to Utz et al. (2015) , 2017, Vision Research.

[86]  E. Daprati,et al.  Unconscious updating of grasp motor program , 2004, Experimental Brain Research.

[87]  M. Goodale,et al.  The visual brain in action , 1995 .

[88]  Tim Shallice,et al.  Supervisory control of action and thought selection. , 1993 .

[89]  Donald A. Norman,et al.  Attention to Action , 1986 .

[90]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .