A STRUCTURED METHODOLOGY FOR COMPARATIVE EVALUATION OF USER INTERFACE DESIGNS USING USABILITY CRITERIA AND MEASURES

Abstract Previous research in software ergonomics has indicated the importance of evaluating the usability of computer user interfaces. This paper presents a quantitative basis for selecting from among multiple alternative interfaces relative to multiple criteria of usability concern. The proposed model consists of two main phases: the prescreening phase and the evaluation phase. The prescreening phase involves expert judgment-based assessment with qualitative criteria. The first phase uses absolute measurement analytic hierarchy process (AHP) to filter possible alternative interfaces to a reasonable subset. The evaluation phase involves user-based assessment such as user testing, with quantitative criteria. The objective of the second phase is to evaluate a subset of alternatives using objective measures and to select the best alternative. A set of criteria and measures for evaluating the usability of computer software designs is presented. The proposed methodology is demonstrated in the interface design of a database system used to analyze trip cases information of nuclear power plant. Relevance to industry There is currently a focus on the usability of interactive computer software. Software developers, interface designers or human factors engineers often confront the task of comparative evaluation among systems, versions or interface designs. The proposed methodology provides practitioners with a structured approach to select the best interface based on usability criteria and measures.

[1]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[2]  Nigel Bevan,et al.  Human-Computer Interaction Standards , 1995 .

[3]  Sidney L. Smith,et al.  Application of Guidelines for Designing User Interface Software , 1985 .

[4]  T. Saaty,et al.  Procedures for Synthesizing Ratio Judgements , 1983 .

[5]  Ron Henderson,et al.  A comparison of the four prominent user-based methods for evaluating the usability of computer software , 1995 .

[6]  V. Belton.  A comparison of the analytic hierarchy process and a simple multi-attribute value function , 1986 .

[7]  S. P. Dutta,et al.  Use of the analytic hierarchy process in ergonomic analysis , 1992 .

[8]  B H Kantowitz,et al.  Selecting Measures for Human Factors Research , 1992, Human factors.

[9]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[10]  Thomas L. Saaty,et al.  Decision Making, Scaling, and Number Crunching , 1989 .

[11]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[12]  Evangelos Triantaphyllou,et al.  USING THE ANALYTIC HIERARCHY PROCESS FOR DECISION MAKING IN ENGINEERING APPLICATIONS: SOME CHALLENGES , 1995 .

[13]  Valerie Belton,et al.  On a short-coming of Saaty's method of analytic hierarchies , 1983 .

[14]  Mansooreh Mollaghasemi,et al.  A composite measure of usability for human-computer interface designs , 1995 .

[15]  Ben Shneiderman,et al.  Designing The User Interface , 2013 .

[16]  D. A. Mitta,et al.  An Application of the Analytic Hierarchy Process: A Rank-Ordering of Computer Interfaces , 1993 .

[17]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[18]  John L. Bennett,et al.  Usability Engineering: Our Experience and Evolution , 1988 .

[19]  Alan L. Tharp,et al.  What users say about software usability , 1991, Int. J. Hum. Comput. Interact..

[20]  Dominique L. Scapin,et al.  Organizing human factors knowledge for the evaluation and design of interfaces , 1990, Int. J. Hum. Comput. Interact..

[21]  Sidney L. Smith,et al.  Guidelines for Designing User Interface Software , 1986 .