User involvement in software development and system success: a systematic literature review

Context: In the last four decades involving users in the software development process is claimed to have a positive impact on the success of that software. However, previous reviews on this topic have produced conflicting results. Objectives: Our aim is to present a review on user involvement in software development process and investigate its relationship to software system success. Methods: For our exploration, we performed a Systematic Literature Review using the guidelines provided in the Evidence Based Software Engineering literature. Results: 87 relevant empirical studies were selected and reviewed that investigate various perspectives and concepts of user involvement in software development process during the period of 1980--2012. Among 87 studies reviewed, 59 report that user involvement positively contributes to system success, 7 suggest a negative contribution and 21 are uncertain. Conclusions: Our results show an overall positive impact of user involvement on system success. It also suggests that the relationship between user involvement and system success is neither direct nor simple, and it depends on many different factors and conditions surrounding systems development processes.

[1]  Daniel Robey,et al.  User Involvement in Information System Development: A Conflict Model and Empirical Test , 1982 .

[2]  Gabriele Piccoli,et al.  Moving beyond user participation to achieve successful IS design , 2007, CACM.

[3]  Stephan O. den Hengst,et al.  Facilitating user involvement in information system design and development with GSS: the organized crime case , 1995, SIGCPR '95.

[4]  Robert Fildes,et al.  Influence of user participation on DSS use and decision accuracy , 2002 .

[5]  Paul John Steinbart,et al.  The Effects of Explanation Type and User Involvement on Learning from and Satisfaction with Expert Systems , 1998 .

[6]  Craig Standing,et al.  Do Project Manager's Utilise Potential Customers in E-Commerce Developments? , 2004 .

[7]  C. R. Franz,et al.  ORGANIZATIONAL CONTEXT, USER INVOLVEMENT, AND THE USEFULNESS OF INFORMATION SYSTEMS* , 1986 .

[8]  Mark I. Hwang,et al.  The effect of user engagement on system success: A meta-analytical integration of research findings , 1999, Inf. Manag..

[9]  Bruce Cheek,et al.  Does user involvement during information systems development improve data quality , 1998 .

[10]  Kieran Conboy,et al.  Distributed Agile Development: A Case Study of Customer Communication Challenges , 2009, XP.

[11]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[12]  Kwasi Amoako-Gyampah,et al.  User involvement and user satisfaction: An exploratory contingency model , 1993, Inf. Manag..

[13]  Peter Johnson,et al.  Helping and hindering user involvement — a tale of everyday design , 1997, CHI.

[14]  Richard Berntsson-Svensson,et al.  Successful software project and products: An empirical investigation , 2006, ISESE '06.

[15]  Leon A. Kappelman,et al.  Measuring user involvement: a diffusion of innovation perspective , 1995, DATB.

[16]  Trevor Wood-Harper,et al.  The finding of thorns: user participation in enterprise system implementation , 2002, DATB.

[17]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[18]  Sylvia Pelayo,et al.  From users involvement to users' needs understanding: A case study , 2010, Int. J. Medical Informatics.

[19]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[20]  William J. Doll,et al.  The Collaborative Use of Information Technology: End-User Participation and Systems Success , 2001, Inf. Resour. Manag. J..

[21]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[22]  Sue Newell,et al.  Exploring the Importance of Participation in the Post-Implementation Period of an ES Project: A Neglected Area , 2007, J. Assoc. Inf. Syst..

[23]  Pascale Carayon,et al.  Sociotechnical issues in the implementation of imaging technology , 2000, Behav. Inf. Technol..

[24]  Henri Barki,et al.  Measuring User Participation, User Involvement, and User Attitude , 1994, MIS Q..

[25]  June M. Verner,et al.  Case study: factors for early prediction of software development success , 2002, Inf. Softw. Technol..

[26]  Melanie Wilson,et al.  Participation: 'Bounded Freedom' or Hidden Constraints on User Involvement , 2003 .

[27]  Geir Kjetil Hanssen,et al.  Agile customer engagement: a longitudinal qualitative case study , 2006, ISESE '06.

[28]  Mark Staples,et al.  Experiences using systematic review guidelines , 2006, J. Syst. Softw..

[29]  Sid L. Huff,et al.  Implementing Decision Support Systems: Correlates Of User Satisfaction And System Usage , 1990 .

[30]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[31]  James E. Hunton,et al.  Involving Information System Users in Defining System Requirements: The Influence of Procedural Justice Perceptions on User Attitudes and Performance , 1996 .

[32]  George M. Marakas,et al.  The Impact of Operational User Participation on Perceived System Implementation Success: An Empirical Investigation , 2006, J. Comput. Inf. Syst..

[33]  S. Thomas Foster,et al.  On the differences between information systems' users and analysts: Managing perceptions of systems quality , 1998 .

[34]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[35]  Quey-Jen Yeh,et al.  Two conflict potentials during IS development , 2001, Inf. Manag..

[36]  James Noble,et al.  Agile Undercover: When Customers Don't Collaborate , 2010, XP.

[37]  Claes Wohlin.  Agile Processes in Software Engineering and Extreme Programming , 2012, Lecture Notes in Business Information Processing.

[38]  Kathy Brittain White,et al.  Information Systems Development Success: Perspectives from Project Team Participants , 1986, MIS Q..

[39]  Mark Keil,et al.  The user–developer communication process: a critical case study , 2003, Inf. Syst. J..

[40]  Jinjoo Lee,et al.  An exploratory contingency model of user participation and MIS use , 1986, Inf. Manag..

[41]  David Graham Wastell,et al.  An information system profile of the UK manufacturing sector , 1995, J. Inf. Technol..

[42]  Marcus O'Connor,et al.  MIS success: why does it vary among users? , 1992, J. Inf. Technol..

[43]  Tor Guimaraes,et al.  The Relationship Between User Participation and User Satisfaction: An Investigation of Four Contingency Factors , 1994, MIS Q..

[44]  T. S. Ragu-Nathan,et al.  Investigating the Level of End-User Development and Involvement Among Time-Based Competitors , 2002, Decis. Sci..

[45]  Michael Frese,et al.  Don't underestimate the problems of user centredness in software development projectsthere are many! , 1996, Behav. Inf. Technol..

[46]  M. Lynne Markus,et al.  Participation in Development and Implementation - Updating An Old, Tired Concept for Today's IS Contexts , 2004, J. Assoc. Inf. Syst..

[47]  William J. Doll,et al.  The measurement of end-user software involvement , 1990 .

[48]  Tore Dybå,et al.  Applying Systematic Reviews to Diverse Study Types: An Experience Report , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[49]  Angèle L. M. Cavaye,et al.  User participation in system development revisited , 1995, Inf. Manag..

[50]  Felix C. Brodbeck,et al.  Communication and performance in software development projects , 2001 .

[51]  Jong-Min Choe,et al.  The effects of user participation on the design of accounting information systems , 1998, Inf. Manag..

[52]  Tone Bratteteig,et al.  User Participation and Democracy: A Discussion of Scandinavian Research on System Development , 1995, Scand. J. Inf. Syst..

[53]  William J. Doll,et al.  A Congruence Construct of User Involvement , 1991 .

[54]  William J. Doll,et al.  The test-retest reliability of user involvement instruments , 1994, Inf. Manag..

[55]  Hsi-Peng Lu,et al.  The relationships between management styles, user participation, and system success over MIS growth stages , 1997, Inf. Manag..

[56]  Netta Iivari,et al.  'Representing the User' in software development - a cultural analysis of usability work in the product development context , 2006, Interact. Comput..

[57]  Anand Vadapalli,et al.  Information technology project outcomes: user participation structures and the impact of organization behavior and human resource management issues , 2000 .

[58]  Björn Johansson.  Diffusion of Open Source ERP Systems Development: How Users Are Involved , 2011, Governance and Sustainability in Information Systems.

[59]  Michael Newman,et al.  User Involvement as an Interaction Process: A Case Study , 1990, Inf. Syst. Res..

[60]  Laura Lehtola,et al.  Investigating the State of User Involvement in Practice , 2009, 2009 16th Asia-Pacific Software Engineering Conference.

[61]  Blake Ives,et al.  Chargeback Systems and User Involvement in Information Systems , 1982, MIS Q..

[62]  Shivraj Kanungo,et al.  Understanding user participation and involvement in ERP use , 2000 .

[63]  James E. Hunton,et al.  Effects of the user participation process and task meaningfulness on key information system outcomes , 1997 .

[64]  Hugh J. Watson,et al.  A Contingency Model for User Involvement in DSS Development , 1984, MIS Q..

[65]  Khaled El Emam,et al.  User participation in the requirements engineering process: An empirical study , 1996, Requirements Engineering.

[66]  James E. Hunton,et al.  User Participation in Defining System Interface Requirements: An Issue of Procedural Justice , 1997 .

[67]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[68]  Ramaraj Palanisamy,et al.  Strategic information systems planning model for building flexibility and success , 2005, Ind. Manag. Data Syst..

[69]  Blake Ives,et al.  Executive Involvement and Participation in the Management of Information Technology , 1991, MIS Q..

[70]  William J. Doll,et al.  Encouraging user management participation in systems design , 1987, Inf. Manag..

[71]  Naveed Saleem,et al.  An Empirical Test of the Contingency Approach to User Participation in Information Systems Development , 1996, J. Manag. Inf. Syst..

[72]  C. Lawrence Meador,et al.  Setting Priorities for DSS Development , 1984, MIS Q..

[73]  Marjo Kauppinen,et al.  The role of user involvement in requirements quality and project success , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).

[74]  James E. Hunton,et al.  Effects of User Participation in Systems Development: A Longitudinal Field Experiment , 1997, MIS Q..

[75]  Maya Daneva,et al.  A comparative case study on clients participation in a 'traditional' and in an Agile software company , 2011, PROFES Workshops.

[76]  Sari Kujala,et al.  User involvement: A review of the benefits and challenges , 2003, Behav. Inf. Technol..

[77]  Benjamin B. M. Shao,et al.  The relationship between user participation and system success: a simultaneous contingency approach , 2000, Inf. Manag..

[78]  Henri Barki,et al.  Communication as a dimension of user participation , 2001 .

[79]  Henri Barki,et al.  Rethinking the Concept of User Involvement , 1989, MIS Q..

[80]  Karlheinz Kautz,et al.  Participatory Design Activities and Agile Software Development , 2010, Human Benefit through the Diffusion of Information Systems Design Science Research.

[81]  Blake Ives,et al.  User Involvement in System Design: An Empirical Test of Alternative Approaches , 1981, Inf. Manag..

[82]  Samuli Pekkola,et al.  Towards formalised end-user participation in information systems development process: bridging the gap between participatory design and ISD methodologies , 2006, PDC '06.

[83]  Stephen R. Hawk,et al.  Successful system development: the effect of situational factors on alternative user roles , 1991 .

[84]  Sari Kujala,et al.  user involvement in product development by improving the analysis of user needs , 2016 .

[85]  Adel M. Aladwani.  IT Planning Effectiveness in a Developing Country , 2001 .

[86]  Ann-Marie K. Baronas,et al.  Restoring a Sense of Control During Implementation: How User Involvement Leads to System Acceptance , 1988, MIS Q..

[87]  Naveed Ikram,et al.  Empirical Evidence in Software Architecture: A Systematic Literature Review Protocol , 2011, ICSEA 2011.

[88]  Tor Guimaraes,et al.  Successful Strategies for User Participation in Systems Development , 1997, J. Manag. Inf. Syst..

[89]  William R. King,et al.  The Role of User Participation in Information Systems Development: Implications from a Meta-Analysis , 2008, J. Manag. Inf. Syst..

[90]  Daniela Cruzes,et al.  Research synthesis in software engineering: A tertiary study , 2011, Inf. Softw. Technol..

[91]  S. T. Foster,et al.  User involvement during information systems development: a comparison of analyst and user perceptions of system acceptance , 1999 .

[92]  A. V. Sioukas,et al.  User involvement for effective customization: an empirical study on voice networks , 1995 .

[93]  Henri Barki,et al.  Explaining the Role of User Participation in Information System Use , 1994 .

[94]  Tor Guimaraes,et al.  Exploring the Factors Associated with Expert Systems Success , 1995, MIS Q..

[95]  Susan Gasson.  User Involvement In Decision-Making In Information Systems Development , 1995 .

[96]  Tor Guimaraes,et al.  Empirically testing the outcomes of user involvement in DSS development , 1994 .

[97]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[98]  Andrew David Irvine.  S7 , 2013, J. Appl. Log..

[99]  William R. King,et al.  Note-Participative Design of Strategic Decision Support Systems: An Empirical Assessment , 1981 .

[100]  Graham C. Low,et al.  Exploring Individual User Satisfaction Within User-Led Development , 1993, MIS Q..

[101]  Dorothy Leonard-Barton,et al.  Developer-User Interaction and User Satisfaction in Internal Technology Transfer , 1993 .