A survey of usability capability maturity models: implications for practice and research

Improving the position and effectiveness of user-centred design (UCD) in software and product development is a challenge in many companies. One step towards improvements is to carry out a usability capability maturity (UCM) assessment to evaluate the strengths and weaknesses of a development organisation in UCD. While several diverse UCM models have been introduced, little research has been published in the public domain on these models. The paper aims to fill this gap by analysing the main features of the models. The results show that most models represent different approaches – although some of them have the same roots – meaning that understanding one model is not enough to understand the basics of another model. All models can be used for examining the status of UCD in individual development projects. In addition, models provide various means for assessment of the status of UCD in other organisational areas. The level of documentation of models varies a lot, and very few empirical research results exist. Based on the results, implications for practice and research are suggested.

[1]  George A. Flanagan.  Usability management maturity, part 1: self assessment—how do you stack up? , 1996, SGCH.

[2]  Pekka Abrahamsson,et al.  New directions on agile methods: a comparative analysis , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[3]  J. Cunningham,et al.  Case study principles for different types of cases , 1997 .

[4]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[5]  Jonathan Earthy,et al.  Usability Maturity Model: Human Centredness Scale , 1998 .

[6]  Chris W. Clegg,et al.  Problems integrating user participation into software development , 1997, Int. J. Hum. Comput. Stud..

[7]  Mark C. Paulk,et al.  The Capability Maturity Model: Guidelines for Improving the Software Process , 1994 .

[8]  Jonathan Earthy,et al.  The Benefits of Using ISO 13407: Human Centred Design Process for Interactive Systems , 2001, INTERACT.

[9]  Deborah J. Mayhew,et al.  The usability engineering lifecycle , 1998, CHI Conference Summary.

[10]  Gerard O’Regan,et al.  The Capability Maturity Model , 2002 .

[11]  J Earthy.  Usability Maturity Model: Processes , 2002 .

[12]  K. Lyytinen.  A taxonomic perspective of information systems development: theoretical constructs and recommendations , 1987 .

[13]  Timo Jokela,et al.  Evaluating the user-centredness of development organisations: conclusions and implications from empirical usability capability maturity assessments , 2004, Interact. Comput..

[14]  Jerry Lake.  The Guide to ISO/IEC 15288 System Life Cycle Processes , 2002 .

[15]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[16]  D. Morgan,et al.  Sociological Paradigms and Organizational Analysis. , 1983 .

[17]  E. Rogers.  Diffusion of Innovations , 1962 .

[18]  Michael E. McGrath,et al.  Setting the PACE in product development : a guide to product and cycle-time excellence , 1996 .

[19]  James W. Moore.  System Life Cycle Processes , 2006 .

[20]  Luo Huai,et al.  System Security Engineering Capability Maturity Model , 2003 .

[21]  Randolph G. Bias,et al.  Cost-justifying usability , 2005 .

[22]  Pasi Kuvaja.  Software Process Assessment and Improvement: The BOOTSTRAP Approach , 1994 .

[23]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[24]  H. Rex Hartson,et al.  Developing user interfaces: ensuring usability through product & process , 1993 .

[25]  Pekka Abrahamsson,et al.  Usability Assessment of an Extreme Programming Project: Close Co-operation with the Customer Does Not Equal to Good Usability , 2004, PROFES.

[26]  P. Coughlan,et al.  Development of a technical innovation audit , 1996 .

[27]  Steve Farner.  Quality is still free: Making quality certain in uncertain times , 1996 .

[28]  Robert Szakonyi,et al.  Measuring R&D Effectiveness—I , 1994 .

[29]  A. Tenbrunsel,et al.  Organizational Behavior and Human Decision Processes , 2013 .

[30]  James Moultrie,et al.  Exploratory studies of a proposed Design Maturity Model , 2001 .

[31]  Rudy Hirschheim,et al.  A Comparison of Five Alternative Approaches to Information Systems Development , 1997, Australas. J. Inf. Syst..

[32]  Jakob Nielsen,et al.  Chapter 6 – Usability Testing , 1993 .

[33]  Timo Jokela,et al.  Developing A Usability Capability Assessment Approach through Experiments in Industrial Settings , 2001, BCS HCI/IHM.

[34]  James Backhouse,et al.  Current directions in IS security research: towards socio‐organizational perspectives , 2001, Inf. Syst. J..

[35]  Richard J. Anderson,et al.  Organizational limits to HCI: conversations with Don Norman and Janice Rohn , 2000, INTR.

[36]  Sarah A. Bloomer,et al.  Successful strategies for selling usability into organisations , 1999, CHI EA '99.

[37]  Peter Johnson,et al.  Helping and hindering user involvement — a tale of everyday design , 1997, CHI.

[38]  Mike Gregory,et al.  A maturity grid approach to the assessment of product development collaborations , 2002 .

[39]  A. F. Chalmers,et al.  What Is This Thing Called Science , 1976 .

[40]  Jakob Nielsen,et al.  What makes strategic usability fail?: lessons learned from the field , 1999, CHI EA '99.

[41]  Juhani Iivari,et al.  Object-oriented information systems analysis: a framework for object identification , 1991, Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences.

[42]  Kate Ehrlich,et al.  Cost justification of usability engineering: a vendors's perspective , 1994 .

[43]  Rudy Hirschheim,et al.  A Paradigmatic Analysis Contrasting Information Systems Development Approaches and Methodologies , 1998, Inf. Syst. Res..

[44]  Rudy Hirschheim,et al.  Analyzing Information Systems Development a Comparison and Analysis of Eight IS Development Approaches , 1996, Inf. Syst..

[45]  Juhani Iivari,et al.  A Sociocybernetic Framework for the Feature Analysis of Information Systems Design Methodologies , 1983, CRIS.

[46]  Aarnout Brombacher,et al.  Analysis of quality information flows in the product creation process of high-volume consumer products , 2000 .

[47]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[48]  I. Ajzen.  The theory of planned behavior , 1991 .

[49]  Thyra L. Rauch,et al.  Usability management maturity, part 1 (abstract): self assessment—how do you stack up? , 1995, CHI '95.

[50]  Philip B. Crosby,et al.  Quality Is Free: The Art of Making Quality Certain , 1979 .

[51]  Mike Gregory,et al.  The use of maturity models/grids as a tool in assessing product development capability , 2002, IEEE International Engineering Management Conference.

[52]  Timo Jokela.  Making user-centred design common sense: striving for an unambiguous and communicative UCD process model , 2002, NordiCHI '02.

[53]  M. Bishop.  What is this thing called Science , 1996 .

[54]  Patty Curthoys,et al.  Developing user interfaces: Ensuring usability through product and process , 1997 .

[55]  Timo Jokela,et al.  Usability and CMMI: does a higher maturity level in product development mean better usability? , 2003, CHI Extended Abstracts.

[56]  Timo Jokela,et al.  Assessment of user-centred design processes as a basis for improvement action , 2001 .

[57]  Richard Baskerville,et al.  Diversity in information systems action research methods , 1998 .

[58]  Rudy Hirschheim,et al.  Crisis in the IS Field? A Critical Reflection on the State of the Discipline , 2003, J. Assoc. Inf. Syst..

[59]  Timo Jokela,et al.  Assessing Human-Centred Design Processes in Product Development by Using the INUSE Maturity Model , 1998 .

[60]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[61]  Raghu Singh.  ISO/IEC draft international standard 12207, software life-cycle processes , 1994, Methods and Associated Tools for the Information Systems Life Cycle.

[62]  Cmmi Product Team.  Capability Maturity Model® Integration (CMMI), Version 1.1--Staged Representation , 2002 .

[63]  David E. Avison,et al.  The fiction of methodological development: a field study of information systems development , 1999, Inf. Technol. People.

[64]  Karel Vredenburg,et al.  A survey of user-centered design practice , 2002, CHI.

[65]  Geoff Walsham,et al.  Interpretive case studies in IS research: nature and method , 1995 .

[66]  Timo Jokela,et al.  The KESSU Usability Design Process Model , 2004 .