Does time heal?: a longitudinal study of usability

We report from a longitudinal laboratory-based usability evaluation of an interactive system. A usability evaluation was conducted with novice users when a large commercial electronic patient record system was being deployed in the use organization. After the users had used the system in their daily work for 15 months, same evaluation was conducted again. Our aim was to inquire into the nature of usability problems experienced by novice and expert users over time, and to see to what extends usability problems may or may not disappear over time, as users get more familiar with the system. On the basis of our two usability evaluations, we present key findings on the usability of the evaluated system as experienced by the two categories of users at these two different points in time. Based on our findings, we discuss implications for evaluating usability.

[1]  Dieter Zapf,et al.  Errors in computerized office work: differences between novice and expert users , 1991, SGCH.

[2]  Gregory D. Abowd,et al.  Human-Computer Interaction, third edition , 2004 .

[3]  Robert W. Bailey,et al.  Usability Testing vs. Heuristic Evaluation: A Head-to-Head Comparison , 1992 .

[4]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[5]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[6]  Morten Hertzum,et al.  The evaluator effect in usability tests , 1998, CHI Conference Summary.

[7]  Wendy A. Kellogg,et al.  Evaluating user and system models: applying scaling techniques to problems in human-computer interaction , 1986, CHI '87.

[8]  Kazuhisa Miwa,et al.  Interactive processes between mental and external operations in creative activity: a comparison of experts' and novices' performance , 2002, Creativity & Cognition.

[9]  Patricia Q. Bourie,et al.  Usability Evaluation of an On-Line Nursing Assessment , 1997, AMIA.

[10]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[11]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[12]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[13]  Bryce Allen,et al.  Cognitive Abilities and Information System Usability , 1994, Inf. Process. Manag..

[14]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[15]  Andrew Dillon,et al.  An empirical comparison of the usability for novice and expert searchers of a textual and a graphic interface to an art-resource database , 1997, J. Digit. Inf..

[16]  Kenichi Tanaka,et al.  NEM: "novice expert ratio method" a usability evaluation method to generate a new performance measure , 2000, CHI Extended Abstracts.

[17]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[18]  Markku Tukiainen,et al.  Effects of display blurring on the behavior of novices and experts during program debugging , 2005, CHI Extended Abstracts.