User involvement: A review of the benefits and challenges

User involvement is a widely accepted principle in development of usable systems. However, it is a vague concept covering many approaches. This study first clarifies the nature of user involvement and its expected benefits, and secondly reviews three streams of research, to evaluate the benefits and problems of varied user involvement approaches in practice. The particular focus of this study is on the early activities in the development process. An analysis of the literature suggests that user involvement has generally positive effects, especially on user satisfaction, and some evidence exists to suggest that taking users as a primary information source is an effective means of requirements capture. However, the role of users must be carefully considered and more cost-efficient practices are needed for gathering users' implicit needs and requirements in real product development contexts.

[1]  Marilyn Tremaine,et al.  Cost/benefit analysis for incorporating human factors in the software lifecycle , 1988, CACM.

[2]  Heather W. Desurvire,et al.  Faster, cheaper!! Are usability inspection methods as effective as empirical testing? , 1994 .

[3]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[4]  Arnold M. Lund.  Another approach to justifying the cost of usability , 1997, INTR.

[5]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[6]  Dianne Juhl.  Using field-oriented design techniques to develop consumer software products , 1996 .

[7]  Pelle Ehn,et al.  Scandinavian Design: On Participation and Skill , 1992, Usability - Turning Technologies into Tools.

[8]  Clare-Marie Karat,et al.  A comparison of user interface evaluation methods , 1994 .

[9]  MariAnne Karlsson,et al.  User Requirements Elicitation - A Framework for the Study of the Relation between User and Artefact , 1996 .

[10]  Mathilde M. Bekker,et al.  User Involvement in the Design of Human - Computer Interactions: Some Similarities and Differences between Design Approaches , 2000, BCS HCI.

[11]  Mary Beth Butler.  Getting to know your users: usability roundtables at Lotus Development , 1996, INTR.

[12]  Erran Carmel,et al.  PD and joint application design: a transatlantic comparison , 1993, CACM.

[13]  Robin Jeffries,et al.  The Role of Task Analysis in the Design of Software , 1997 .

[14]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[15]  Jonathan Grudin,et al.  Organizational obstacles to interface design and development: two participant-observer studies , 1994, TCHI.

[16]  David E. Rowley.  Organizational considerations in field-oriented product development: experiences of a cross-functional team , 1996 .

[17]  Tom Rodden,et al.  The role of ethnography in interactive systems design , 1995, INTR.

[18]  Sara A. Bly,et al.  Field work: is it product work? , 1997, INTR.

[19]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[20]  Janice Redish,et al.  Task analysis , 2002 .

[21]  Dennis R. Wixon.  Qualitative research methods in design and development , 1995, INTR.

[22]  Jakob Nielsen,et al.  Chapter 4 – The Usability Engineering Lifecycle , 1993 .

[23]  Martti Mäntylä,et al.  How Effective Are User Studies? , 2000, BCS HCI.

[24]  Clare-Marie Karat,et al.  Cost-Justifying Usability Engineering in the Software Life Cycle , 1997 .

[25]  Dan Shapiro,et al.  The limits of ethnography: combining social sciences for CSCW , 1994, CSCW '94.

[26]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[27]  John D. Gould,et al.  The 1984 Olympic Message System: a test of behavioral principles of system design , 1987, CACM.

[28]  Scott Lewis,et al.  Ethnographic data for product development: a collaborative process , 1996, INTR.

[29]  Randolph G. Bias,et al.  Cost-justifying usability , 2005 .

[30]  Jesper Simonsen,et al.  Using ethnography in contextural design , 1997, CACM.

[31]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[32]  John D. Gould,et al.  How to design usable systems , 1995 .

[33]  Ian Sommerville,et al.  Coherence: an approach to representing ethnographic analyses in systems design , 1999 .

[34]  Chauncey E. Wilson,et al.  The Usability Engineering Framework for Product Design and Evaluation , 1997 .

[35]  Erran Carmel,et al.  Customer-developer links in software development , 1995, CACM.

[36]  Michael G. Kahn,et al.  Using contextual inquiry to discover physicians' true needs , 1996 .

[37]  Henri Barki,et al.  User participation and user involvement in information system development , 1991, Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences.

[38]  Dennis Wixon,et al.  Field methods casebook for software design , 1996 .

[39]  Yvonne Rogers,et al.  What Are Workplace Studies For? , 1995, ECSCW.

[40]  Neil A. M. Maiden,et al.  ACRE: selecting methods for requirements acquisition , 1996, Softw. Eng. J..

[41]  David R. Millen,et al.  Rapid ethnography: time deepening strategies for HCI field research , 2000, DIS '00.

[42]  Ian Sommerville,et al.  Social analysis in the requirements engineering process: from ethnography to method , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).

[43]  S. Dray,et al.  A day in the life of a family: an international ethnographic study , 1996 .

[44]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[45]  Karen Holtzblatt,et al.  Contextual techniques starter kit , 1996, INTR.

[46]  Kristin Bauersfeld,et al.  “You've got three days!” Case studies in field techniques for the time-challenged , 1996 .

[47]  Larry E. Wood,et al.  Semi-structured interviewing for user-centered design , 1997, INTR.

[48]  Tor Guimaraes,et al.  Successful Strategies for User Participation in Systems Development , 1997, J. Manag. Inf. Syst..

[49]  Barry Kirwan,et al.  A Guide To Task Analysis: The Task Analysis Working Group , 1992 .

[50]  Michael Frese,et al.  Don't underestimate the problems of user centredness in software development projectsthere are many! , 1996, Behav. Inf. Technol..

[51]  Deborah J. Mayhew,et al.  A basic framework for costs-justifying usability engineering , 1994 .

[52]  Ronald M. Baecker,et al.  Readings in human-computer interaction : toward the year 2000 , 1995 .

[53]  Jonathan Grudin,et al.  Interactive systems: bridging the gaps between developers and users , 1991, Computer.

[54]  Thomas C. Ormerod,et al.  The Role of Task Analysis in Capturing Requirements for Interface Design , 1998, Interact. Comput..

[55]  Alan H. Rowberg,et al.  Adaptation of an ethnographic method for investigation of the task domain in diagnostic radiology , 1992 .

[56]  James E. Hunton,et al.  Effects of User Participation in Systems Development: A Longitudinal Field Experiment , 1997, MIS Q..

[57]  Douglas Schuler,et al.  Participatory Design: Principles and Practices , 1993 .

[58]  Mark Notess,et al.  Customer-focused design data in a large, multi-site organization , 1999, CHI '99.

[59]  Kim Halskov,et al.  Participatory design in Britain and North America: responses to the “Scandinavian Challenge” , 1991, CHI '91.

[60]  Clare-Marie Karat,et al.  Human factors cost justification for an internal development project , 1994 .

[61]  Lucy A. Suchman,et al.  Reflections on a Work-Oriented Design Project , 1996, Hum. Comput. Interact..

[62]  Peter Johnson,et al.  Costs and Benefits of User Involvement in Design: Practitioners' Views , 1996, BCS HCI.

[63]  S. T. Foster,et al.  User involvement during information systems development: a comparison of analyst and user perceptions of system acceptance , 1999 .

[64]  Robert D. Macredie,et al.  The Importance of Context in Information System Design: An Assessment of Participatory Design , 1999, Requirements Engineering.

[65]  A. Shepherd,et al.  Guide to Task Analysis , 2003 .

[66]  Tom Dayton,et al.  Participatory Practices in the Software Lifecycle , 1997 .

[67]  Morten Kyng.  Scandanavian design: users in product development , 1994, CHI Conference Companion.

[68]  Jonathan Grudin,et al.  Systematic Sources of Suboptimal Interface Design in Large Product Development Organizations , 1991, Hum. Comput. Interact..

[69]  Karen Holtzblatt,et al.  Making customer-centered design work for teams , 1993, CACM.

[70]  Daniel Lafrenière.  CUTA: a simple, practical, low-cost approach to task analysis , 1996, INTR.

[71]  John D. Gould.  Chapter 35 – How to Design Usable Systems , 1988 .

[72]  Kate Ehrlich,et al.  Cost justification of usability engineering: a vendors's perspective , 1994 .

[73]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[74]  Linda A. Macaulay,et al.  Requirements capture and analysis: A survey of current practice , 1996, Requirements Engineering.

[75]  Paul Dourish,et al.  On "Technomethodology": Foundational Relationships Between Ethnomethodology and System Design , 1998, Hum. Comput. Interact..

[76]  Peter Johnson,et al.  Helping and hindering user involvement — a tale of everyday design , 1997, CHI.

[77]  Leela Damodaran,et al.  User involvement in the systems design process-a practical guide for users , 1996, Behav. Inf. Technol..

[78]  Gerhard Schmidt,et al.  Out of Scandinavia: Alternative Approaches to Software Design and System Development , 1989, Hum. Comput. Interact..

[79]  Dennis R. Wixon,et al.  Changing the Rules: A Pragmatic Approach to Product Development , 1993, Digit. Tech. J..

[80]  Jonathan Kies,et al.  User and task analysis for interface design , 1998 .

[81]  Clive Frankish,et al.  User involvement in the early stages of the development of an aircraft warning system , 1996, Behav. Inf. Technol..

[82]  Jesper Simonsen,et al.  MUST: A Method for Participatory Design , 1998, Hum. Comput. Interact..

[83]  John Karat,et al.  Evolving the scope of user-centered design , 1997, CACM.

[84]  Dan Shapiro,et al.  Faltering from ethnography to design , 1992, CSCW '92.

[85]  Andrew Clement,et al.  A retrospective look at PD projects , 1993, CACM.

[86]  Diane S. Brown.  The challenges of user-based design in a medical equipment market , 1996 .

[87]  Stephen R. Hawk,et al.  Successful system development: the effect of situational factors on alternative user roles , 1991 .

[88]  Michael Good,et al.  Participatory design of a portable torque-feedback device , 1992, CHI.

[89]  Peter Checkland,et al.  Soft Systems Methodology in Action , 1990 .

[90]  Chris Rockwell.  Customer connection creates a winning product: building success with contextual techniques , 1999, INTR.

[91]  Sara Jones,et al.  Usability for fun and profit: a case study of the design of DEC Rally Version 2 , 1995 .

[92]  Gary D. Scudder,et al.  Concurrent software development , 2000, CACM.

[93]  Michael J. Muller,et al.  Participatory design , 1993, CACM.

[94]  Naveed Saleem,et al.  An Empirical Test of the Contingency Approach to User Participation in Information Systems Development , 1996, J. Manag. Inf. Syst..

[95]  Michael J. Muller,et al.  Using the CARD and PICTIVE participatory design methods for collaborative analysis , 1996 .

[96]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.