Person, place, and past influence eye movements during visual search

Person, place, and past influence eye movements during visual search Barbara Hidalgo-Sotelo (bhs@mit.edu) Aude Oliva (oliva@mit.edu) Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 Abstract features directs gaze to expectation-based scene regions (Eckstein, Drescher & Shimozaki, 2006; Henderson, 2003; Torralba, Oliva, Castelhano, & Henderson, 2006). At the level of scene exemplar representations, spatial context can also be used to allocate attention preferentially to regions that have become associated with the target. In contextual cueing, for example, observers implicitly learn patterns in repeated displays that help them find a target faster in repeated configurations (Chun & Jiang, 1998). It is not well understood, however, whether a scene exemplar representation can systematically bias individual fixations. How can “experience based” influences be distinguished from the myriad of sources that guide attention to relevant scene regions? One challenge is that attention is strongly guided by information that does not depend on specific experience. Figure 1 illustrates regularities in eye fixations across and within observers. In Figure 1A, fixations from 9 observers searching for a book are shown; the high fixation density along countertop surfaces illustrates how spatial layout and context guide where observers look. Systematic biases unrelated to the scene’s content also influence gaze location. In Figure 1B, fixations sampled from random scenes have been projected onto the kitchen scene. Center bias in the fixation distribution is driven by oculomotor tendencies (Tatler, 2007; Tatler & Vincent, 2009) and photographer bias. A second challenge, of the opposing nature, lies in the significant variability in fixation locations across individuals. As a result, two independent observers may fixate different scene regions, even when looking for the same object in the same scene (Figure 1C). It is possible that individuals are biased by experience, but that the effects are masked by pooling over experienced observers. Given initial differences in search patterns, could systematic differences arise when an observer repeats her search of the scene? To reasonably estimate the influence of past experience, the search patterns of observers who have never viewed the scene must be contrasted with different observers who have previously searched the scene. In this paper, eye movement data from a visual search study was analyzed using approach we have termed comparative map analysis. This analysis was used to evaluate how different sources of information contribute to attentional guidance during visual search of familiar scenes. In our experiment, observers’ eyes were tracked while they looked for a book in pictures of real world scenes. On some trials, observers searched a scene that had been presented previously. Importantly, the target object and location remained unchanged in each presentation of the scene. The What is the role of an individual’s past experience in guiding gaze in familiar environments? Contemporary models of search guidance suggest high level scene context is a strong predictor of where observers search in realistic scenes. Specific associations also develop between particular places and object locations. Together, scene context and place- specific associations bias attention to informative spatial locations. At the level of eye fixations, it is not known whether a person’s specific search experience influences attentional selection. Eye movements are notoriously variable: people often foveate different places when searching for the same target in the same scene. Do individual differences in fixation locations influence how a scene is subsequently examined? We introduce a method, comparative map analysis, for analyzing spatial patterns in eye movement data. Using this method, we quantified the consistency of fixated locations within the same observer and between observers during search of real world scenes. Results indicated a remarkable consistency in the locations fixated by the same observer across multiple searches of a given scene. This observer-specific guidance was shown to be distinct from general scene context information or familiarity with the scene. Accordingly, this is considered evidence for a uniquely informative role of an individual’s search experience on attentional guidance in a familiar scene. Keywords: visual search; eye movements; scene perception; learning; attentional guidance; fixation similarity Introduction An important feature of ecological visual search is that there are few truly novel, unfamiliar places in which a person is likely to search. Many tasks involve examining the same place repeatedly, such as the various times spent searching for a specific utensil in one’s own kitchen. Locating the target in question benefits from both category based information (e.g. utensils are on countertops) and place specific information (e.g. in this kitchen, utensils hang over the stove). For any observer, there will be many sources of information that guide which scene regions are inspected during search. What influence does a person’s own search experience (i.e. fixation locations) have in guiding where they are likely to look in familiar scenes? A growing body of evidence suggests that observers use high level information, such as learned target features and global scene context, to guide their gaze when searching for an object in real world environments (Ehinger, Hidalgo- Sotelo, Torralba, & Oliva, 2009; Hwang, Higgins, Pomplun, 2009). At this level of categorical representation, knowledge of the basic-level scene category and target