Novice heuristic evaluations of a complex interface

In this paper, we describe the ability of evaluators with limited experience to use Heuristic Evaluation (HE) in assessing a complex interface. We analyze our results in terms of the proportion of problems found by different sets of evaluators in different areas of the interface. Our results illustrate that the 5-10 evaluator size advocated by Nielsen and Molich [3] does not generalize to assessing complex interfaces. Evaluators tend to focus on certain sections of the interface and ignore others. Our results suggest that modification to HE is necessary to most efficiently produce a complete set of usability problems in an interface.