Assessment of user-centred design processes as a basis for improvement action

Many software-intensive products and systems on the market today reveal a poor level of usability despite the availability of knowledge about how to develop good usability, namely user-centred design. Improving the status of user-centred design has proved to be a challenge in product development. An effective step to start out on the road to such improvements is to carry out a current state analysis. Our research problem was to learn how to perform an effective current state analysis, i.e. an assessment, of user-centred design processes to provide a basis for improvement action. As the main part of our research, we carried out five experimental assessments in industrial settings during the years 2000 and 2001. The main result of the research is a novel assessment approach. The objective of the approach is to provide a basis for improvement of the performance of user-centred design in individual product development projects. The particular features of the approach are a new user-centred design process model, a three-dimensional process performance model and implementation of the assessment in the form of a workshop with extensive participation by the project members. The user-centred design process model is method-independent, and consists of six processes that are defined through outcomes. The performance of processes is assessed from the viewpoints of quantity, quality, and integration. Our theoretical examinations show that different assessment categories can be identified depending on the different foci and purposes of an assessment. Moreover, it may even be appropriate to have different assessment approaches for different categories. In the category of our focus and purpose, i.e. 'assessment of user-centred design processes for performance improvement', the focus of an assessment should be in the substance rather than in the management of UCD processes. An assessment approach should be regarded as an artefact and it should be subject to ongoing refinement. An assessment should be an intervention that provides both an effective training session and a meaningful experience to the participants. Clarity in both the concepts and the results seems to take precedence over having standard reference models. In addition, we created a preliminary theory of usability capability. It identifies three dimensions of usability capability: user-centred design infrastructure, performance of user-centred design in product development projects and usability in business strategy. We also propose some new assessment artefacts for other assessment categories and contrast our research experience with established research frameworks.

[1]  George A. Flanagan.  Usability management maturity, part 1: self assessment—how do you stack up? , 1996, SGCH.

[2]  Deborah J. Mayhew,et al.  The usability engineering lifecycle , 1998, CHI Conference Summary.

[3]  S. B. Kiselev,et al.  The capability maturity model: guidelines for improving the software process , 1995 .

[4]  Timo Jokela,et al.  Assessing Human-Centred Design Processes in Product Development by Using the INUSE Maturity Model , 1998 .

[5]  Pekka Abrahamsson,et al.  Modelling Usability Capability - Introducing the Dimensions , 2000, PROFES.

[6]  Leslie G. Tudor.  Human factors: does your management hear you? , 1998, INTR.

[7]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[8]  Jonathan Earthy,et al.  The improvement of human-centred processes-facing the challenge and reaping the benefit of ISO 13407 , 2001, Int. J. Hum. Comput. Stud..

[9]  Jakob Nielsen,et al.  What makes strategic usability fail?: lessons learned from the field , 1999, CHI EA '99.

[10]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[11]  Timo Jokela,et al.  Using Quantitative Usability Goals in the Design of a User Interface for A Cellular Phone , 1999 .

[12]  Stephanie Rosenbaum,et al.  A toolkit for strategic usability: results from workshops, panels, and surveys , 2000, CHI.

[13]  Timo Jokela,et al.  PERFORMING A USER-CENTRED DESIGN PROCESS ASSESSMENT IN A SMALL COMPANY – HOW THE COMPANY PERCEIVED THE RESULTS , 2000 .

[14]  Tom Carney.  Fourth Generation Evaluation , 1991 .

[15]  Timo Jokela.  Usability Capability Models - Review and Analysis , 2000, BCS HCI.

[16]  Jonathan Earthy,et al.  Usability Maturity Model: Human Centredness Scale , 1998 .

[17]  Sarah A. Bloomer,et al.  Successful strategies for selling usability into organisations , 1999, CHI EA '99.

[18]  Gilbert Cockton,et al.  People and Computers XIV — Usability or Else! , 2000, Springer London.

[19]  Lars Mathiassen,et al.  Formalizations in systems development , 1986 .

[20]  Kate Ehrlich,et al.  Cost justification of usability engineering: a vendors's perspective , 1994 .

[21]  Pertti Järvinen,et al.  Research Questions Guiding Selection of an Appropriate Research Method , 2000, ECIS.

[22]  S. Hakiel.  Sufficient and necessary conditions for routine deployment of user-centred design , 1999 .

[23]  Jonathan Earthy,et al.  A Human Factors Integration Capability Maturity Model , 1999 .

[24]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[25]  Ramkrishnan V. Tenkasi,et al.  P ERSPECTIVE M AKING AND P ERSPECTIVE T AKING IN C OMMUNITIES OF K NOWING , 2000 .

[26]  Timo Jokela,et al.  Developing A Usability Capability Assessment Approach through Experiments in Industrial Settings , 2001, BCS HCI/IHM.

[27]  Philip B. Crosby,et al.  Quality Is Free: The Art of Making Quality Certain , 1979 .

[28]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[29]  Timo Jokela.  Review of Usability Capability Assessment Approaches , 2001 .

[30]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[31]  Timo Jokela,et al.  Evaluating a Usability Capability Assessment , 2001, PROFES.

[32]  H. Rex Hartson,et al.  Developing user interfaces: ensuring usability through product & process , 1993 .

[33]  Deborah J. Mayhew.  Business: Strategic development of the usability engineering function , 1999, INTR.

[34]  Carol H. Weiss.  Evaluation Research: Methods of Assessing Program Effectiveness , 1972 .