Evaluation in Information Visualization: Heuristic Evaluation

This paper presents a review of heuristic evaluation and recommendations for how to apply the method for information visualization evaluation. Heuristic evaluation is a widely known and popular method within the area of human-computer interaction and the information visualization community now also recognizes its usefulness. However, in this area it is not applied to the same extent. In its original form the method has limitations that need to be considered in order for it to be optimal for information visualization. The aim with this paper is to provide the reader with knowledge about the method and awareness of what issues that call for refined or supplemental actions and resources in order for it to generate as valid and useful results as possible. The paper also discusses the research challenges for future work in how to further improve the method.

[1]  B. Marx.  The Visual Display of Quantitative Information , 1985 .

[2]  M. Sheelagh T. Carpendale,et al.  Heuristics for information visualization evaluation , 2006, BELIV '06.

[3]  John Stasko,et al.  BEST PAPER: A Knowledge Task-Based Framework for Design and Evaluation of Information Visualizations , 2004 .

[4]  M. Sheelagh T. Carpendale,et al.  Evaluating Information Visualizations , 2008, Information Visualization.

[5]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[6]  Chris North,et al.  An insight-based methodology for evaluating bioinformatics visualizations , 2005, IEEE Transactions on Visualization and Computer Graphics.

[7]  Ebba Þóra Hvannberg,et al.  Analysis of strategies for improving and estimating the effectiveness of heuristic evaluation , 2004, NordiCHI '04.

[8]  Victoria Interrante,et al.  User Studies: Why, How, and When? , 2003, IEEE Computer Graphics and Applications.

[9]  Karel Vredenburg,et al.  A survey of user-centered design practice , 2002, CHI.

[10]  Dominique L. Scapin,et al.  Ergonomic criteria for evaluating the ergonomic quality of interactive systems , 1997, Behav. Inf. Technol..

[11]  P. R. G. Luzzardi,et al.  An Extended Set of Ergonomic Criteria for Information Visualization Techniques , 2003 .

[12]  Camilla Forsell,et al.  An heuristic set for evaluation in information visualization , 2010, AVI.

[13]  Camilla Forsell,et al.  Voyage Analysis Applied to Geovisual Analytics , 2008, 2008 12th International Conference Information Visualisation.

[14]  David H. Laidlaw,et al.  Thoughts on User Studies: Why, How and When , 1993 .

[15]  Luciana P. Nedel,et al.  Evaluating Usability of Information Visualization Techniques , 2002 .

[16]  Allison Woodruff,et al.  Guidelines for using multiple views in information visualization , 2000, AVI '00.

[17]  Keith Andrews,et al.  Evaluating information visualisations , 2006, BELIV '06.

[18]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[19]  Melanie Tory,et al.  Evaluating Visualizations: Do Expert Reviews Work? , 2005, IEEE Computer Graphics and Applications.

[20]  Gilbert Cockton,et al.  Sale must end: should discount methods be cleared off HCI's shelves? , 2002, INTR.

[21]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[22]  John Stasko.  Evaluating Information Visualizations : Issues and Opportunities , 2006 .

[23]  Deborah Hix,et al.  Usability Evaluation in Virtual Environments: Classification and Comparison of Methods , 2001 .

[24]  L. Faulkner.  Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[25]  Doug A. Bowman,et al.  A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods , 2002, Presence: Teleoperators & Virtual Environments.

[26]  Katharina Kaiser,et al.  CareCruiser: Exploring and visualizing plans, events, and effects interactively , 2011, 2011 IEEE Pacific Visualization Symposium.

[27]  Anind K. Dey,et al.  Heuristic evaluation of ambient displays , 2003, CHI '03.

[28]  Mary Czerwinski,et al.  Empirical evaluation of information visualizations: an introduction , 2000, Int. J. Hum. Comput. Stud..

[29]  Jeffrey Heer,et al.  Animated Transitions in Statistical Data Graphics , 2007, IEEE Transactions on Visualization and Computer Graphics.

[30]  M. Sheelagh T. Carpendale,et al.  Theoretical analysis of uncertainty visualizations , 2006, Electronic Imaging.

[31]  Jakob Nielsen,et al.  Enhancing the explanatory power of usability heuristics , 1994, CHI '94.

[32]  Tom L. Roberts,et al.  Improving the Usability Evaluation Technique, Heuristic Evaluation, through the Use of Collaborative Software , 2005, AMCIS.

[33]  Gilbert Cockton,et al.  Why and when five test users aren’t enough , 2001 .

[34]  Paul A. Cairns,et al.  Beyond guidelines: what can we learn from the visual information seeking mantra? , 2005, Ninth International Conference on Information Visualisation (IV'05).

[35]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[36]  T. Griffin.  Semiology of graphics: diagrams, networks, maps. Bertin, Jacques (trans. W. j. Berg). The University of Wisconsin Press, Madison, 1983. 416 pages. ISBN 0 299 09060 4. US$75.00, cloth. , 1987 .

[37]  Robert Spence,et al.  Information Visualization: Design for Interaction (2nd Edition) , 2006 .

[38]  Jared M. Spool,et al.  Testing web sites: five users is nowhere near enough , 2001, CHI Extended Abstracts.

[39]  Wolfgang Aigner,et al.  A-plan: integrating interactive visualization with automated planning for cooperative resource scheduling , 2011, i-KNOW '11.

[40]  Amitava Datta,et al.  A parallel coordinates style interface for exploratory volume visualization , 2005, IEEE Transactions on Visualization and Computer Graphics.

[41]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .