Mental imagery in problem solving: an eye tracking study

Cognitive models and empirical studies of problem solving in visuo-spatial and causal domains suggest that problem solving tasks in such domains invoke cognitive processes involving mental animation and imagery. If these internal processes are externally manifested in the form of eye movements, such tasks present situations in which the trajectory of a user's visual attention can provide clues regarding his or her information needs to an Attentive User Interface [Vertegaal 2002]. In this paper, we briefly review research related to problem solving that involves mental imagery, and describe an experiment that looked for evidence and effects of an imagery strategy in problem solving. We eye-tracked 90 subjects solving two causal reasoning problems, one in which a diagram of the problem appeared on the stimulus display, and a second related problem that was posed on a blank display. Results indicated that 42% of the subjects employed mental imagery and visually scanned the display in a correspondingly systematic fashion. This suggests that information displays that respond to a user's visual attention trajectory, a kind of Attentive User Interface, are more likely to benefit this class of users.

[1]  Richard K. Lowe.  Extracting information from an animation during complex visual learning , 1999 .

[2]  N. Hari Narayanan,et al.  Multimedia design for communication of dynamic information , 2002, Int. J. Hum. Comput. Stud..

[3]  Richard A. Bolt,et al.  A gaze-responsive self-disclosing display , 1990, CHI '90.

[4]  Robert J. K. Jacob,et al.  Eye Movement-Based Human-Computer Interaction Techniques: Toward Non-Command Interfaces , 2003 .

[5]  M. Hegarty,et al.  Individual differences in mental animation during mechanical reasoning , 1994, Memory & cognition.

[6]  N. Hari Narayanan,et al.  Reactive Information Displays , 2003, INTERACT.

[7]  M. Hegarty.  Mental animation: inferring motion from static displays of mechanical systems. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[8]  N. Hari Narayanan,et al.  Reasoning Visually about Spatial Interactions , 1991, IJCAI.

[9]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[10]  Roel Vertegaal,et al.  Attentive User Interfaces , 2003 .

[11]  Daniel L. Schwartz,et al.  Analog Imagery in Mental Model Reasoning: Depictive Models , 1996, Cognitive Psychology.

[12]  Roel Vertegaal,et al.  Designing attentive interfaces , 2002, ETRA.

[13]  R. Shepard,et al.  Mental Images and Their Transformations , 1982 .

[14]  Michael J. Spivey,et al.  Mechanical Reasoning about Gear-and-belt Diagrams: Do Eye-movements Predict Performance? , 2002, Diagrammatic Representation and Reasoning.

[15]  N. Hari Narayanan,et al.  On designing comprehensible interactive hypermedia manuals , 1998, Int. J. Hum. Comput. Stud..