A model-based framework for classifying and diagnosing usability problems

A great deal of study has been devoted to the problem of how to identify and categorize usability problems; however, there is still a lack of studies dealing with the problem of how to diagnose the causes of usability problems and how to feed them back into design process. The value of classifying usability problems can be enhanced when they are interpreted in connection with design process and activities. Thus, it is necessary to develop a systematic way of diagnosing usability problems in terms of design aspects and applying diagnosis results to improve design process and activities. With this issue in mind, this paper proposes a conceptual framework that supports a systematic classification and diagnosis of usability problems. This paper firstly reviews seven approaches to classifying usability problems. Then, we point out the needs of adopting a model-based approach to classifying and diagnosing usability problems and of developing a comprehensive framework guiding the use of model-based approaches. We then propose a conceptual framework that specifies how a model-based classification and diagnosis of usability problems should be conducted and suggests the combined use of three different types of models, each of which addresses context of use, design knowledge and design activities. Last, we explain how a sound classification scheme of usability problems can be systematically developed, and how the classification of usability problems can be connected to design process and activities on the basis of the framework.

[1]  Bernd G. Freimut Developing and using defect classification schemes , 2001 .

[2]  Andrea J. van Doorn,et al.  Usability Problem Reports for Comparative Studies: Consistency and Inspectability , 2008, Hum. Comput. Interact..

[3]  Gilsoo Cho,et al.  Development of a usability evaluation framework with quality function deployment: From customer sensibility to product design , 2009 .

[4]  Kasper Hornbæk,et al.  Dogmas in the assessment of usability evaluation methods , 2010, Behav. Inf. Technol..

[5]  Sheue-Ling Hwang,et al.  Design for usability on supply chain management systems implementation , 2009 .

[6]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[7]  Dan Saffer,et al.  Designing for Interaction: Creating Smart Applications and Clever Devices , 2006 .

[8]  Philippe Kruchten,et al.  Casting software design in the function-behavior-structure framework , 2005, IEEE Software.

[9]  Steven M. Belz,et al.  The user action framework: a reliable foundation for usability engineering support tools , 2001, Int. J. Hum. Comput. Stud..

[10]  Virginia Tech,et al.  A Structured Process for Transforming Usability Data into Usability Information , 2007 .

[11]  Silvia Mara Abrahão,et al.  Usability evaluation methods for the web: A systematic mapping study , 2011, Inf. Softw. Technol..

[12]  Erik Hollnagel,et al.  Cognitive reliability and error analysis method , 1998 .

[13]  Wolmet Barendregt,et al.  A comparison of two analytical evaluation methods for educational computer games for young children , 2007, Cognition, Technology & Work.

[14]  Kasper Hornbæk,et al.  Current practice in measuring usability: Challenges to usability studies and research , 2006, Int. J. Hum. Comput. Stud..

[15]  Malcolm P. Atkinson,et al.  Comparison of evaluation methods using structured usability problem reports , 1997, Behav. Inf. Technol..

[16]  Paula Savioja,et al.  Systems usability framework for evaluating tools in safety–critical work , 2013, Cognition, Technology & Work.

[17]  N CardDavid Learning from Our Mistakes with Defect Causal Analysis , 1998 .

[18]  Yvonne Rogers,et al.  Interaction Design - Beyond Human-Computer Interaction, 3rd Edition , 2012 .

[19]  Harry Budi Santoso,et al.  Measuring the user experience , 2008 .

[20]  Kim J. Vicente,et al.  Ecological Interface Design: Progress and Challenges , 2002, Hum. Factors.

[21]  Mark V. Springett,et al.  Linking surface error characteristics to root problems in user-based evaluation studies , 1998, AVI '98.

[22]  Guy A. Boy,et al.  Cognitive Function Analysis , 1998 .

[23]  N. Naikar,et al.  Analysing activity in complex systems with cognitive work analysis: concepts, guidelines and case study for control task analysis , 2006 .

[24]  Henrik Artman,et al.  Finding a way to usability: procurement of a taxi dispatch system , 2005, Cognition, Technology & Work.

[25]  Robert Scane,et al.  User Interface Design , 2003 .

[26]  Dennis G. Kafura,et al.  The Usability Problem Taxonomy: A Framework for Classification and Analysis , 1999, Empirical Software Engineering.

[27]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[28]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[29]  David N. Card,et al.  Learning from Our Mistakes with Defect Causal Analysis , 1999, IEEE Softw..

[30]  Dov Te'eni,et al.  Human-Computer Interaction: Developing Effective Organizational Information Systems , 2006 .

[31]  Ann Blandford,et al.  Models of Interactive Systems: a Case Study on Programmable User Modelling Models of Interactive Systems: a Case Study on Programmable User Modelling , 2022 .

[32]  Nigel Bevan,et al.  Quality in use: Meeting user needs for quality , 1999, J. Syst. Softw..

[33]  Inderpal S. Bhandari,et al.  Orthogonal Defect Classification - A Concept for In-Process Measurements , 1992, IEEE Trans. Software Eng..

[34]  Hokyoung Ryu,et al.  Analysing interaction problems with cyclic interaction theory: Low-level interaction walkthrough , 2004, PsychNology J..

[35]  Jeongyun Heo,et al.  A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors , 2009, Interact. Comput..

[36]  Gitte Lindgaard,et al.  Notions of thoroughness, efficiency, and validity: Are they valid in HCI practice? , 2006 .

[37]  G. Salvendy,et al.  Human Factors and Ergonomics in Manufacturing & Service Industries , 2009 .

[38]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[39]  Douglas J. Gillan,et al.  Usability Science. I: Foundations , 2001, Int. J. Hum. Comput. Interact..

[40]  Adinda Freudenthal,et al.  The evaluation of an innovative intelligent thermostat interface: universal usability and age differences , 2003, Cognition, Technology & Work.

[41]  Jon T. Huber A Comparison of IBM ’ s Orthogonal Defect Classification to Hewlett Packard ’ s Defect Origins , Types , and Modes 1 . 0 , 1999 .

[42]  Ebba Þóra Hvannberg,et al.  Classification of usability problems (CUP) scheme: augmentation and exploitation , 2006, NordiCHI '06.

[43]  Erik Hollnagel,et al.  The Phenotype of Erroneous Actions , 1993, Int. J. Man Mach. Stud..

[44]  Dewayne E. Perry,et al.  Classification and evaluation of defects in a project retrospective , 2002, J. Syst. Softw..

[45]  Lesley Strawderman,et al.  Human factors and usability in service quality measurement , 2008 .

[46]  Kasper Hornbæk,et al.  Comparing usability problems and redesign proposals as input to practical systems development , 2005, CHI.

[47]  John S. Gero,et al.  The Situated Function — Behaviour — Structure Framework , 2004 .

[48]  John S. Gero,et al.  The Situated Function - Behaviour - Structure Framework , 2002, AID.

[49]  Jens Rasmussen,et al.  The role of hierarchical knowledge representation in decisionmaking and system management , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[50]  Marc Hassenzahl Prioritizing usability problems: Data-driven and judgement-driven severity estimates , 2000, Behav. Inf. Technol..

[51]  Wan Chul Yoon,et al.  Evaluation of software usability using scenarios organized by abstraction structure , 2007, ECCE '07.

[52]  B. Schneirdeman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .