Gaze strategies in object identification and manipulation

Gaze strategies in object identification and manipulation Anna Belardinelli (belardinelli@informatik.uni-tuebingen.de) Department of Computer Science, University of T¨ubingen, Martin V. Butz (martin.butz@uni-tuebingen.de) Department of Computer Science, Department of Psychology, University of T¨ubingen, Sand 14, T¨ubingen, 72076 Germany Abstract Task influence has long been known to play a major role in the way our eyes scan a scene. Interestingly, how the task modu- lates attention when interacting with objects has been less in- vestigated. Only few studies have contrasted the distribution of eye fixations during viewing and grasping objects. How is at- tention differently deployed when different actions have to be planned on objects in contrast to a purely perceptual viewing condition? To investigate these issues, we conducted an eye- tracking experiment showing participants 2D images of real- world objects. In blocks of trials, participants were asked ei- ther to assign the displayed objects to one of two classes (clas- sification task), to mimic lifting the object (lifting task), or to mimic opening the object (opening task). Mean fixation lo- cations and attention heatmaps show different modes in gaze distribution around task-relevant locations, in accordance with previous literature. Reaction times, measured by button release in the manual response, suggest that the more demanding the task in terms of motor planning the longer the latency in move- ment initiation. Results show that even on simplified, two di- mensional displays the eyes reveal the current intentions of the participants. Moreover, the results suggest elaborate cognitive processes at work and confirm anticipatory behavioral control. We conclude with suggesting that the strongly predictive in- formation contained in eye movements data may be used for advanced, highly intuitive, user-friendly brain-computer inter- faces. Keywords: Eye-tracking, object interaction, fixation distribu- tion, eye-hand coordination, movement preparation Introduction Since the early works of Buswell (1935) and Yarbus (1967) top-down, task-related guidance has been shown to strongly influence the way people move their gaze upon pictures. In the second study, depending on the question asked, differ- ent patterns of scanning were observed. Such an influence is so critical that, as soon as a specific task is given, low-level, bottom-up visual saliency is basically overridden and plays quite a minor role in explaining eye fixations w.r.t. higher- level cognitive factors (Henderson, Brockmole, Castelhano, & Mack, 2007; Einh¨auser, Rutishauser, & Koch, 2008). Sim- ilarly, moving from pictures to real-world scenes and to tasks involving motor actions, it is even more striking how eye movements are precisely planned to provide information for the execution of the current piece of action. This has been shown in different settings, from tea-making (Land, Mennie, & Rusted, 1999) to sandwich-making (Hayhoe, Shrivastava, Mruczek, & Pelz, 2003) to a wealth of other more or less complex motor tasks (Land & Tatler, 2009). In this case, any- way, the nature of attention deployment is quite different. The purpose of vision is here indeed less to get sense of the scene and more to direct effectors and coordinate a much slower and more complex behaviour than scanning. Strategies like ’look-ahead’ and ’just-in-time’ fixations (Hayhoe et al., 2003; Ballard, Hayhoe, & Pelz, 1995) support the idea that vision is deeply intertwined with the needs of motion planning and supervising. Further, in the context of the theory on the duplex nature of vision (Goodale & Milner, 1992), distinct neural pathways subserving the different functional demands of object cate- gorization and object manipulation were suggested. This dis- sociation between vision-for-action and vision-for-perception has often been investigated by means of grasping tasks con- trasted to perceptual judgement tasks, with visual illusions or in covert attention settings (Goodale, 2011), but contrasting evidence has emerged and it seems reasonable to assume a strict interaction between the two systems. How the differences between perceptual and motor task are reflected in eye-movements has been less investigated. In a seminal paper for eye-hand coordination, Johansson, West- ling, Backstrom, and Flanagan (2001) recorded both eye- and hand movements data during a motor task involving grasping a bar, avoiding an obstacle, touching a goal position and plac- ing the bar back. Subjects almost exclusively fixated land- mark positions on the bar or in the experimental set-up, be- fore making contact to them. The preparation of an action upon an object defines an attentional landscape (Baldauf & Deubel, 2010), (covertly) encoding in parallel locations rele- vant for the subsequent serial motor execution. This evidence suggests that visual cues are sought and weighted differently depending if the task is a skilled move- ment or a perceptual judgement. Gaze behaviour in viewing and grasping was investigated by (Brouwer, Franz, & Gegen- furtner, 2009) and (Desanghere & Marotta, 2011). The first ones used simple geometric shapes to be simply viewed or grasped, while in the latter study Efron blocks were used and in the viewing condition a perceptual judgement had to be made. In both cases, the viewing condition produced first fixations closer to the center-of-gravity (COG) of the object (in accordance with (Foulsham & Underwood, 2009), among others), while the grasping condition was characterized by first fixations closer to the index finger location (or to the more difficult to grasp location). In this paper, we present an experiment building on that of Brouwer et al. (2009). The main novelty of our approach is the use of real object stimuli (displayed on a monitor) and the comparison of three simple but realistic tasks, one ’pas- sive’ (classification) and two ’active’ (lifting and opening).