Text or Pictures? An Eyetracking Study of How People View Digital Video Surrogates

One important user-oriented facet of digital video retrieval research involves how to abstract and display digital video surrogates. This study reports on an investigation of digital video results pages that use textual and visual surrogates. Twelve subjects selected relevant video records from results lists containing titles, descriptions, and three keyframes for ten different search tasks. All subjects were eye-tracked to determine where, when, and how long they looked at text and image surrogates. Participants looked at and fixated on titles and descriptions statistically reliably more than on the images. Most people used the text as an anchor from which to make judgments about the search results and the images as confirmatory evidence for their selections. No differences were found whether the layout presented text or images in left to right order.

[1]  Gerald L. Lohse,et al.  Consumer Eye Movement Patterns on Yellow Pages Advertising , 1997 .

[2]  Phillip J. Moore,et al.  Verbal and visual learning styles , 1988 .

[3]  Robert J. K. Jacob,et al.  What you look at is what you get: eye movement-based interaction techniques , 1990, CHI '90.

[4]  Gary Marchionini,et al.  The Open Video Digital Library , 2002, D Lib Mag..

[5]  Alistair G. Sutcliffe,et al.  Making contact points between text and images , 1998, MULTIMEDIA '98.

[6]  D. Krugman,et al.  Adolescents' Attention to Beer and Cigarette Print Ads and Associated Product Warnings , 1998 .

[7]  Boris M. Velichkovsky,et al.  Visual fixations and level of attentional processing , 2000, ETRA.

[8]  M. Hegarty.  Mental animation: inferring motion from static displays of mechanical systems. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[9]  Andrew J. Stewart,et al.  Integrating text and pictorial information: eye movements when looking at print advertisements. , 2001, Journal of experimental psychology. Applied.

[10]  Jeff B. Pelz,et al.  Extended tasks elicit complex eye movement patterns , 2000, ETRA.

[11]  Mary Hegarty,et al.  The Mechanics of Comprehension and Comprehension of Mechanics , 1992 .

[12]  K. Rayner.  Eye movements and visual cognition : scene perception and reading , 1992 .

[13]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[14]  Gary Marchionini,et al.  Alternative Surrogates for Video Objects in a Digital Library: Users' Perspectives on Their Relative Usability , 2002, ECDL.

[15]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.