What the Draughtsman's Hand Tells the Draughtsman's Eye: a Sensorimotor Account of Drawing

In this paper we address the challenging problem of sensorimotor integration, with reference to eye-hand coordination of an artificial agent engaged in a natural drawing task. Under the assumption that eye-hand coupling influences observed movements, a motor continuity hypothesis is exploited to account for how gaze shifts are constrained by hand movements. A Bayesian model of such coupling is presented in the form of a novel Dynamic Bayesian Network, namely an Input‐Output Coupled Hidden Markov Model. Simulation results are compared to those obtained by eye-tracked human subjects involved in drawing experiments.

[1]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[2]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[4]  Christopher M. Brown,et al.  Controlling eye movements with hidden Markov models , 2004, International Journal of Computer Vision.

[5]  S R Ellis,et al.  Statistical Dependency in Visual Scanning , 1986, Human factors.

[6]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[7]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[8]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[9]  D H Ballard,et al.  Hand-eye coordination during sequential tasks. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[10]  Hagai Attias,et al.  Planning by Probabilistic Inference , 2003, AISTATS.

[11]  Emma Gowen,et al.  Eye-hand interactions in tracing and drawing tasks. , 2006, Human movement science.

[12]  Yoshua Bengio,et al.  Input-output HMMs for sequence processing , 1996, IEEE Trans. Neural Networks.

[13]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[14]  G. Rizzolatti,et al.  Effects of spatial attention on directional manual and ocular responses , 1997, Experimental Brain Research.

[15]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[16]  Laura Dempere-Marco,et al.  Eye movement and voluntary control in portrait drawing , 2003 .

[17]  Gary Feng,et al.  Eye movements as time-series random variables: A stochastic model of eye movement control in reading , 2006, Cognitive Systems Research.

[18]  Lawrence W. Stark,et al.  Visual perception and sequences of eye movement fixations: a stochastic modeling approach , 1992, IEEE Trans. Syst. Man Cybern..

[19]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[20]  Paolo Napoletano,et al.  The Bayesian Draughtsman: A Model for Visuomotor Coordination in Drawing , 2007, BVAI.