Expert vs Novice Evaluators - Comparison of Heuristic Evaluation Assessment

In this paper authors propose the comparison between results of website heuristic evaluation performed by small group of experts and large group of novice evaluators. Normally, heuristic evaluation is performed by few experts and requires their knowledge and experience to apply heuristics effectively. However, research involving usability experts is usually very costly. Therefore, authors propose the experiment in order to contrast the results of evaluation performed by novice evaluators that are familiar with assessed website to the results obtained from expert evaluators in order to verify if they are comparable. The usability of website was evaluated using the authors’ heuristics with extended list of control questions.

[1]  David G. Novick,et al.  Usability inspection methods after 15 years of research and practice , 2007, SIGDOC '07.

[2]  Kasper Hornbæk,et al.  Metaphors of human thinking for usability inspection and design , 2008, TCHI.

[3]  Andrew Dillon,et al.  An empirical comparison of the usability for novice and expert searchers of a textual and a graphic interface to an art-resource database , 1997, J. Digit. Inf..

[4]  Gavriel Salvendy,et al.  Effect of evaluators' cognitive style on heuristic evaluation: Field dependent and field independent evaluators , 2009, Int. J. Hum. Comput. Stud..

[5]  Dieter Zapf,et al.  Errors in computerized office work: differences between novice and expert users , 1991, SGCH.

[6]  Thomas K. Landauer,et al.  Trouble with Computers: Usefulness, Usability, and Productivity , 1996 .

[7]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[8]  Antonio Peñalver Benavent,et al.  A new proposal for improving heuristic evaluation reports performed by novice evaluators , 2013, ChileCHI '13.

[9]  P. Kidwell,et al.  The trouble with computers: Usefulness, usability and productivity , 1996, IEEE Annals of the History of Computing.

[10]  Susan Weinschenk,et al.  Designing effective speech interfaces , 2000 .

[11]  Marek Milosz,et al.  Memorability Experiment Vs. Expert Method in Websites Usability Evaluation , 2013, ICEIS.

[12]  Jill Gerhardt‐Powals,et al.  Cognitive engineering principles for enhancing human-computer performance , 1996, Int. J. Hum. Comput. Interact..

[13]  Morten Hertzum,et al.  The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[14]  Ben Shneiderman,et al.  Research-Based Web Design & Usability Guidelines [2006 edition] , 2006 .

[15]  Antonella De Angeli,et al.  Do patterns help novice evaluators? A comparative study , 2011, Int. J. Hum. Comput. Stud..