Eyecatch: simulating visuomotor coordination for object interception

We present a novel framework for animating human characters performing fast visually guided tasks, such as catching a ball. The main idea is to consider the coordinated dynamics of sensing and movement. Based on experimental evidence about such behaviors, we propose a generative model that constructs interception behavior online, using discrete submovements directed by uncertain visual estimates of target movement. An important aspect of this framework is that eye movements are included as well, and play a central role in coordinating movements of the head, hand, and body. We show that this framework efficiently generates plausible movements and generalizes well to novel scenarios.

[1]  Robert Sessions Woodworth,et al.  THE ACCURACY OF VOLUNTARY MOVEMENT , 1899 .

[2]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[3]  Laurence R. Young,et al.  Variable Feedback Experiments Testing a Sampled Data Model for Eye Tracking Movements , 1963 .

[4]  D. Robinson.  The mechanics of human smooth pursuit eye movement. , 1965, The Journal of physiology.

[5]  Emilio Bizzi,et al.  The coordination of eye and head movement during smooth pursuit , 1978, Brain Research.

[6]  S. McKee.  A local mechanism for differential velocity detection , 1981, Vision Research.

[7]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  D. Robinson,et al.  The upper limit of human smooth pursuit velocity , 1985, Vision Research.

[9]  J. F. Soechting,et al.  Organization of arm movements. Motion is segmented , 1987, Neuroscience.

[10]  Jean-Jacques E. Slotine,et al.  Experiments in Robotic Catching , 1991, 1991 American Control Conference.

[11]  J. Wessberg,et al.  Organization of motor output in slow finger movements in man. , 1993, The Journal of physiology.

[12]  Demetri Terzopoulos,et al.  Artificial fishes: physics, locomotion, perception, behavior , 1994, SIGGRAPH.

[13]  Peter J. Beek,et al.  The Science of Juggling , 1995 .

[14]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[15]  R. H. S. Carpenter,et al.  Neural computation of log likelihood in control of saccadic eye movements , 1995, Nature.

[16]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 1995, Biological Cybernetics.

[17]  Demetri Terzopoulos,et al.  Animat vision: Active vision in artificial animals , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  D. Sparks,et al.  Combined eye-head gaze shifts produced by electrical stimulation of the superior colliculus in rhesus monkeys. , 1996, Journal of neurophysiology.

[19]  S. Schaal,et al.  Segmentation of endpoint trajectories does not imply segmented control , 1999, Experimental Brain Research.

[20]  Neil A. Dodgson,et al.  Ball Catching: an example of psychologically-based behavioural animation , 1999 .

[21]  N. Shimizu.  [Neurology of eye movements]. , 2000, Rinsho shinkeigaku = Clinical neurology.

[22]  F. Lacquaniti,et al.  Does the brain model Newton's laws? , 2001, Nature Neuroscience.

[23]  R. Johansson,et al.  Eye–Hand Coordination in Object Manipulation , 2001, The Journal of Neuroscience.

[24]  Edward G. Freedman,et al.  Interactions between eye and head control signals can account for movement kinematics , 2001, Biological Cybernetics.

[25]  Digby Elliott,et al.  A ménage À trois: the eye, the hand and on-line processing , 2002, Journal of sports sciences.

[26]  N. Badler,et al.  Eyes Alive Eyes Alive Eyes Alive Figure 1: Sample Images of an Animated Face with Eye Movements , 2022 .

[27]  K. E. Novak,et al.  The use of overlapping submovements in the control of rapid hand movements , 2002, Experimental Brain Research.

[28]  Christopher G. Atkeson,et al.  Robot Catching: Towards Engaging Human-Humanoid Interaction , 2002, Auton. Robots.

[29]  Neville Hogan,et al.  Avoiding spurious submovement decompositions: a globally optimal algorithm , 2003, Biological Cybernetics.

[30]  Laurent Itti,et al.  Realistic avatar eye and head animation using a neurobiological model of visual attention , 2004, SPIE Optics + Photonics.

[31]  J. Crawford,et al.  Neural control of three-dimensional eye and head movements , 2003, Current Opinion in Neurobiology.

[32]  Mel Slater,et al.  The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment , 2003, CHI '03.

[33]  James Park,et al.  The Brain's Sense of Movement , 2003, The Yale Journal of Biology and Medicine.

[34]  Katsu Yamane,et al.  Synthesizing animations of human manipulation tasks , 2004, ACM Trans. Graph..

[35]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, ACM Trans. Graph..

[36]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 2004, Biological Cybernetics.

[37]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[38]  J. L. Gordon,et al.  A model of the smooth pursuit eye movement system , 1986, Biological Cybernetics.

[39]  Demetri Terzopoulos,et al.  Autonomous pedestrians , 2005, SCA '05.

[40]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[41]  Daniel Bullock,et al.  How Position, Velocity, and Temporal Information Combine in the Prospective Control of Catching: Data and Model , 2005, Journal of Cognitive Neuroscience.

[42]  Jaroslaw Francik,et al.  Character animation with decoupled behaviour and smart objects , 2005 .

[43]  Eugene Fiume,et al.  Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion , 2005, SCA '05.

[44]  Victor B. Zordan,et al.  Physically based grasping control from example , 2005, SCA '05.

[45]  Jovan Popovic,et al.  Interactive animation of dynamic manipulation , 2006, SCA '06.

[46]  L. Itti.  Quantitative modelling of perceptual salience at human eye position , 2006 .

[47]  Demetri Terzopoulos,et al.  Heads up!: biomechanical modeling and neuromuscular control of the neck , 2006, ACM Trans. Graph..

[48]  Norman I. Badler,et al.  Visual Attention and Eye Gaze During Multiparty Conversations with Distractions , 2006, IVA.

[49]  David J. Fleet,et al.  Multifactor Gaussian process models for style-content separation , 2007, ICML '07.

[50]  Aaron Hertzmann,et al.  Active learning for real-time motion controllers , 2007, ACM Trans. Graph..

[51]  Jean-Jacques Orban de Xivry,et al.  Saccades and pursuit: two outcomes of a single sensorimotor process , 2007, The Journal of physiology.

[52]  F. Lacquaniti,et al.  Visuo-motor coordination and internal models for object interception , 2009, Experimental Brain Research.

[53]  David J. Fleet,et al.  Topologically-constrained latent variable models , 2008, ICML '08.

[54]  Dinesh K. Pai,et al.  Musculotendon simulation for hand animation , 2008, ACM Trans. Graph..

[55]  N. Hogan,et al.  Submovement changes characterize generalization of motor recovery after stroke , 2009, Cortex.

[56]  C. Karen Liu,et al.  Dextrous manipulation from a grasping pose , 2009, ACM Trans. Graph..

[57]  Eftychios Sifakis,et al.  Comprehensive biomechanical modeling and simulation of the upper body , 2009, TOGS.

[58]  Daniel Thalmann,et al.  Simulating gaze attention behaviors for crowds , 2009, Comput. Animat. Virtual Worlds.

[59]  Christopher E. Peters,et al.  A head movement propensity model for animating gaze shifts and blinks of virtual characters , 2010, Comput. Graph..

[60]  Berthold Bäuml,et al.  Realtime perception for catching a flying ball with a mobile humanoid , 2011, 2011 IEEE International Conference on Robotics and Automation.

[61]  Alexander Dietrich,et al.  Catching flying balls and preparing coffee: Humanoid Rollin'Justin performs dynamic and sensitive tasks , 2011, 2011 IEEE International Conference on Robotics and Automation.