Eye–Hand Coordination in Object Manipulation

We analyzed the coordination between gaze behavior, fingertip movements, and movements of the manipulated object when subjects reached for and grasped a bar and moved it to press a target-switch. Subjects almost exclusively fixated certain landmarks critical for the control of the task. Landmarks at which contact events took place were obligatory gaze targets. These included the grasp site on the bar, the target, and the support surface where the bar was returned after target contact. Any obstacle in the direct movement path and the tip of the bar were optional landmarks. Subjects never fixated the hand or the moving bar. Gaze and hand/bar movements were linked concerning landmarks, with gaze leading. The instant that gaze exited a given landmark coincided with a kinematic event at that landmark in a manner suggesting that subjects monitored critical kinematic events for phasic verification of task progress and subgoal completion. For both the obstacle and target, subjects directed saccades and fixations to sites that were offset from the physical extension of the objects. Fixations related to an obstacle appeared to specify a location around which the extending tip of the bar should travel. We conclude that gaze supports hand movement planning by marking key positions to which the fingertips or grasped object are subsequently directed. The salience of gaze targets arises from the functional sensorimotor requirements of the task. We further suggest that gaze control contributes to the development and maintenance of sensorimotor correlation matrices that support predictive motor control in manipulation.

[1]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[2]  D. Robinson.  The mechanics of human saccadic eye movement , 1964, The Journal of physiology.

[3]  R. Steinman.  Effect of Target Size, Luminance, and Color on Monocular Fixation* , 1965 .

[4]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[5]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[6]  R. Baloh,et al.  Quantitative measurement of saccade amplitude, duration, and velocity , 1975, Neurology.

[7]  Lawrence Stark,et al.  Glissades—eye movements generated by mismatched components of the saccadic motoneuronal control signal , 1975 .

[8]  R. Carpenter,et al.  Movements of the Eyes , 1978 .

[9]  K. Rayner.  Eye movements in reading and information processing. , 1978, Psychological bulletin.

[10]  L. Stark,et al.  Glissadic overshoots are due to pulse width errors. , 1978, Archives of neurology.

[11]  J. Findlay.  Global visual processing for saccadic eye movements , 1982, Vision Research.

[12]  W. Wolf,et al.  Corrective saccades: Effect of shifting the saccade goal , 1982, Vision Research.

[13]  F. Ottes,et al.  Metrics of saccade responses to visual double stimuli: Two different modes , 1984, Vision Research.

[14]  W. Wolf,et al.  The Evaluation of the Oculomotor Error Signal , 1984 .

[15]  D. Robinson,et al.  Saccadic undershoot is not inevitable: Saccades can be accurate , 1986, Vision Research.

[16]  H. Collewijn,et al.  Binocular co‐ordination of human horizontal saccadic eye movements. , 1988, The Journal of physiology.

[17]  H. Collewijn,et al.  Differences in accuracy of human saccades between stationary and jumping targets , 1989, Vision Research.

[18]  P. Viviani.  Eye movements in visual search: cognitive, perceptual and motor control aspects. , 1990, Reviews of oculomotor research.

[19]  E Kowler,et al.  Saccadic localization of eccentric forms. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[20]  A. Inhoff,et al.  Encoding of text, manual movement planning, and eye-hand coordination during copytyping. , 1992, Journal of experimental psychology. Human perception and performance.

[21]  Michael F. Land,et al.  Predictable eye-head coordination during driving , 1992, Nature.

[22]  D H Ballard,et al.  Hand-eye coordination during sequential tasks. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[23]  A. Inhoff,et al.  Encoding of text, manual movement planning, and eye-hand coordination during copytyping. , 1992, Journal of experimental psychology. Human perception and performance.

[24]  J. Vickers.  Gaze Control in Putting , 1992, Perception.

[25]  K. Rayner.  Eye movements and visual cognition : scene perception and reading , 1992 .

[26]  Mary Hayhoe,et al.  Spatial Memory and Integration Across Saccadic Eye Movements , 1992 .

[27]  K. J. Cole,et al.  Memory representations underlying motor commands used during manipulation of common and novel objects. , 1993, Journal of neurophysiology.

[28]  Eileen Kowler,et al.  Slow control with eccentric targets: Evidence against a position-corrective model , 1993, Vision Research.

[29]  John M. Findlay,et al.  The spatial signal for saccadic eye movements emphasizes visual boundaries , 1993, Perception & psychophysics.

[30]  Thomas W. Goolsby.  Eye Movement in Music Reading: Effects of Reading Ability, Notational Complexity, and Encounters , 1994 .

[31]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[32]  R S Johansson,et al.  Grasp stability during manipulative actions. , 1994, Canadian journal of physiology and pharmacology.

[33]  Jeremiah Y. Cohen,et al.  The neural basis of saccade target selection , 1995 .

[34]  R. Carpenter,et al.  Saccadic eye movements while reading music , 1995, Vision Research.

[35]  Michael Land,et al.  Which parts of the road guide steering? , 1995, Nature.

[36]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[37]  Erik Blaser,et al.  The accuracy and precision of saccades to small and large targets , 1995, Vision Research.

[38]  Paul Dassonville,et al.  The use of egocentric and exocentric location cues in saccadic programming , 1995, Vision Research.

[39]  D. Robinson,et al.  Shared neural control of attentional shifts and eye movements , 1996, Nature.

[40]  J. Vickers,et al.  Visual control when aiming at a far target. , 1996, Journal of experimental psychology. Human perception and performance.

[41]  M F Land,et al.  The knowledge base of the oculomotor system. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[42]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[43]  M. Hayhoe,et al.  Reference frames in saccadic targeting , 1997, Experimental Brain Research.

[44]  P Jenmalm,et al.  Visual and Somatosensory Information about Object Shape Control Manipulative Fingertip Forces , 1997, The Journal of Neuroscience.

[45]  A E Patla,et al.  Where and when do we look as we approach and step over an obstacle in the travel path? , 1997, Neuroreport.

[46]  Eileen Kowler,et al.  Saccadic localization of random dot targets , 1998, Vision Research.

[47]  D. Pélisson,et al.  From Eye to Hand: Planning Goal-directed Movements , 1998, Neuroscience & Biobehavioral Reviews.

[48]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[49]  G. Bock,et al.  Sensory guidance of movement , 1998 .

[50]  R S Johansson,et al.  Sensory input and control of grip. , 1998, Novartis Foundation symposium.

[51]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[52]  Eileen Kowler,et al.  Shapes, surfaces and saccades , 1999, Vision Research.

[53]  J. Schall,et al.  Neural selection and control of visually guided eye movements. , 1999, Annual review of neuroscience.

[54]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[55]  J. Flanagan,et al.  Independence of perceptual and sensorimotor predictions in the size–weight illusion , 2000, Nature Neuroscience.

[56]  S. Liversedge,et al.  Saccadic eye movements and cognition , 2000, Trends in Cognitive Sciences.

[57]  P Jenmalm,et al.  Visual and tactile information about object-curvature control fingertip forces and grasp kinematics in human dexterous manipulation. , 2000, Journal of neurophysiology.