Decoupling of Intuitions and Performance in the Use of Complex Visual Displays

Decoupling of Intuitions and Performance in the Use of Complex Visual Displays Mary Hegarty 1 (hegarty@psych.ucsb.edu) Harvey S. Smallman 2 (smallman@pacific-science.com) Andrew T. Stull 1 (stull@psych.ucsb.edu) Department of Psychology, University of California, Santa Barbara, CA 93106, USA Pacific Science and Engineering Group, San Diego, CA 92121, USA Worryingly, past research suggests that users often have poor intuitions about what makes effective graphical displays (Smallman & St. John, 2005). In particular, users have misplaced faith in realistic representations and their ability to extract information from them–they mistake familiarity for effectiveness. Smallman and St. John term this misplaced faith Naive Realism. For example, Navy users prefer spatially realistic 3D icons of ships and planes on their displays but these features lead to slow, error-prone identification. Similarly, users predict they will need high fidelity realistic 3D displays to lay routes across terrain when they actually perform the task better with lower fidelity displays that unmask the valleys and other avenues through the terrain necessary to successfully perform the task (Smallman, Cook, Manes, & Cowen, 2007). Meteorology offers a rich domain in which to study these issues. Forecasters use weather maps for a variety of tasks, including reconciling model data with observations, generating forecasts for different client needs, and issuing warnings of severe weather events. The displays that they use while performing these tasks typically show a variety of different meteorological variables (pressure, wind, temperature, etc.) superimposed on the map. Existing display systems give forecasters a great deal of flexibility and tailorability in terms of what variables are shown (Hoffman, Detweiler, Conway, & Lipton, 1993). Given the flexibility they have, how well do users tailor their displays to best show the information they need? In a recent naturalistic observation of twenty one Navy weather forecasters (Smallman & Hegarty, 2007), we found that when performing a forecasting task, participants accessed weather maps that were more complex than they needed, displaying variables that were extraneous to their task. This effect was exacerbated with forecasters of lower spatial ability. That is, low-spatial forecasters put more extraneous variables in their displays and their forecasts were somewhat less accurate. In his cognitive analysis of principles of graphics design, Kosslyn (1989) states as a cardinal rule that “no more or less information should be provided than is needed by the user:” (p. 211). Tufte (1983) also cautions against including extraneous information in visual displays, calling this information “chartjunk”. In adding extraneous variables to their displays, these meteorologists are violating a basic principle of effective graphics. The question we ask here is to what extent these extraneous weather map variables actually impair performance. In a preliminary laboratory study on this question Canham, Hegarty, and Smallman (2007) had participants Abstract Interactive display systems give users flexibility to tailor visual displays to different tasks and situations. But this flexibility can only be beneficial if users have meta- knowledge of the types of displays that are effective for different purposes. In previous studies with both Navy forecasters and undergraduate students, we found that users often prefer maps that display extraneous variables, especially those that add realism to the display, even when they are task- irrelevant. In the current study, we tested undergraduate students on a simple read-off and comparison task with weather maps and measured the effect of these extraneous variables on accuracy, response times, eye fixations, and intuitions about display effectiveness. Extraneous realism slowed response time and lead to more eye fixations on both task-relevant and task-irrelevant regions of the displays. In spite of these decrements in performance, and the fact that realism added no task-relevant information, about a third of participants persisted in favoring these realistic displays over non-realistic maps. Keywords: Visual Displays, Naive Realism, Eye Fixations. Introduction Research on comprehension and reasoning with visual displays has made it clear that the effectiveness of a visual display depends on the task to be performed with that display (Bertin, 1983; Cheng, 2002; Larkin & Simon, 1987; Shah, Freedman & Vekiri, 2005). The same task may be performed more or less efficiently with different types of displays, and a display that is effective for one task may not be useful for another. Given the task-dependent nature of display effectiveness, how can cognitive scientists and designers best support information processing with visual displays? One traditional human factors approach is to analyze the information needs and tasks to be accomplished by particular users and design displays that best support these tasks. This is a good approach for bounded task domains with stable information needs over time. But for users who perform many diverse tasks with different information demands, this may not be feasible. An alternative solution for less stable domains is to give the user control over parameters of the display, so that he or she can customize displays at will. However, it is important to realize that this solution puts the burden of the design process on the user. When users are allowed to customize their own displays, effective performance relies more on their knowledge and intuitions about which displays are most effective for different tasks.