Perspective-based Usability Inspection: An Empirical Validation of Efficacy

Inspection is a fundamental means of achieving software usability. Past research showed that the current usability inspection techniques were rather ineffective. We developed perspective-based usability inspection, which divides the large variety of usability issues along different perspectives and focuses each inspection session on one perspective. We conducted a controlled experiment to study its effectiveness, using a post-test only control group experimental design, with 24 professionals as subjects. The control group used heuristic evaluation, which is the most popular technique for usability inspection. The experimental design and the results are presented, which show that inspectors applying perspective-based inspection not only found more usability problems related to their assigned perspectives, but also found more overall problems. Perspective-based inspection was shown to be more effective for the aggregated results of multiple inspectors, finding about 30% more usability problems for 3 inspectors. A management implication of this study is that assigning inspectors more specific responsibilities leads to higher performance. Internal and external threats to validity are discussed to help better interpret the results and to guide future empirical studies.

[1]  Michael E. Atwood,et al.  What is gained and lost when using evaluation methods other than empirical testing , 1993 .

[2]  Adam A. Porter,et al.  Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment , 1995, IEEE Trans. Software Eng..

[3]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[4]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[5]  Andrew Sears,et al.  Heuristic Walkthroughs: Finding the Problems Without the Noise , 1997, Int. J. Hum. Comput. Interact..

[6]  David R. Schwartz,et al.  The use of guidelines in interface design , 1991, CHI.

[7]  J. E. Sammet,et al.  Software psychology: human factors in computer and information systems , 1983, SGCH.

[8]  Victor R. Basili,et al.  Evolving and packaging reading technologies , 1997, J. Syst. Softw..

[9]  Victor R. Basili,et al.  An Empirical Study of Communication in Code Inspections , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[10]  Ben Shneiderman,et al.  An Empirical Study of Perspective-Based Usability Inspection , 1998 .

[11]  Heather W. Desurvire,et al.  Faster, cheaper!! Are usability inspection methods as effective as empirical testing? , 1994 .

[12]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[13]  Peter G. Polson,et al.  An automated cognitive walkthrough , 1991, CHI '91.

[14]  Jakob Nielsen,et al.  Paper versus computer implementations as mockup scenarios for heuristic evaluation , 1990, INTERACT.

[15]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[16]  Joseph F. Dumas,et al.  A Practical Guide to Usability Testing , 1993 .

[17]  Forrest Shull,et al.  The empirical investigation of Perspective-Based Reading , 1995, Empirical Software Engineering.

[18]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[19]  Peter C. Wright,et al.  A Cost-Effective Evaluation Method for Use by Designers , 1991, Int. J. Man Mach. Stud..

[20]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[21]  Jakob Nielsen,et al.  Assessing the usability of a user interface standard , 1991, CHI.

[22]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[23]  Cathleen Wharton,et al.  The cognitive walkthrough method: a practitioner's guide , 1994 .