Understanding the Influence of User Participation and Involvement on System Success – a Systematic Mapping Study

User participation and involvement in software development are considered to be essential for a successful software system. Three research areas, human aspects of software engineering, requirements engineering, and information systems, study these topics from various perspectives. We think it is important to analyze user participation and involvement in software engineering comprehensively to encourage further research in this area. We investigate the evidence on effects of user participation and involvement on system success and we explore which methods are available in literature. A systematic mapping study was conducted. The systematic search yielded 3,698 hits, from which we identified 289 unique papers. These papers were reviewed by the first author based on inclusion and exclusion criteria. The second author validated the selection of papers by reviewing the reasons for exclusion and inclusion and the corresponding papers on a sample base. 58 of the 289 papers were selected (22 statistical survey and meta-study papers and 36 methods papers). Based on the empirical evidence of the surveys and meta-studies, we developed a meta-analysis of structural equation models. This overview demonstrates that most papers showed positive correlations between aspects of development processes (including user participation) and human aspects (including user involvement) and system success. The analysis of the proposed solutions from the method papers revealed a wide variety of user participation and involvement practices for most activities within software development.

[1]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[2]  Charles A. Gallagher.  Perceptions of the Value of a Management Information System , 1974 .

[3]  C. R. Franz,et al.  ORGANIZATIONAL CONTEXT, USER INVOLVEMENT, AND THE USEFULNESS OF INFORMATION SYSTEMS* , 1986 .

[4]  Daniel Robey,et al.  User Involvement in Information System Development: A Conflict Model and Empirical Test , 1982 .

[5]  Steve Easterbrook,et al.  Communication Problems in Requirements Engineering: A Field Study , 1996 .

[6]  Marjo Kauppinen,et al.  The role of user involvement in requirements quality and project success , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).

[7]  Tor Guimaraes,et al.  Successful Strategies for User Participation in Systems Development , 1997, J. Manag. Inf. Syst..

[8]  James E. Hunton,et al.  Effects of User Participation in Systems Development: A Longitudinal Field Experiment , 1997, MIS Q..

[9]  Andreas Eckhardt,et al.  Lost in translation?!: the need for a boundary spanner between business and it , 2010, SIGMIS-CPR '10.

[10]  Jeff Sutherland,et al.  Manifesto for Agile Software Development , 2013 .

[11]  Jungwoo Lee,et al.  Measures of perceived end-user computing skills , 2003, Inf. Manag..

[12]  Kieran Conboy,et al.  Combining Agile and Traditional: Customer Communication in Distributed Environment , 2010, Agility Across Time and Space.

[13]  S. T. Foster,et al.  User involvement during information systems development: a comparison of analyst and user perceptions of system acceptance , 1999 .

[14]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[15]  Juhani Iivari,et al.  The user – the great unknown of systems development: reasons, forms, challenges, experiences and intellectual contributions of user involvement , 2010, Inf. Syst. J..

[16]  Zahir Irani,et al.  Analyzing the role of stakeholders in the adoption of technology integration solutions in UK local government: An exploratory study , 2011, Gov. Inf. Q..

[17]  Cynthia M. Jackson,et al.  Toward an Understanding of the Behavioral Intention to Use an Information System , 1997 .

[18]  Claes Wohlin,et al.  Systematic literature studies: Database searches vs. backward snowballing , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[19]  William R. King,et al.  Note-Participative Design of Strategic Decision Support Systems: An Empirical Assessment , 1981 .

[20]  Brian Fitzgerald,et al.  A case study of user participation in the information systems development process , 1997, ICIS '97.

[21]  Juhani Iivari,et al.  Determinants of user participation: A Finnish survey , 1997, Behav. Inf. Technol..

[22]  Blake Ives,et al.  Executive Involvement and Participation in the Management of Information Technology , 1991, MIS Q..

[23]  C WetherbeJames,et al.  The relationship between user participation and user satisfaction , 1994 .

[24]  Mark I. Hwang,et al.  The effect of user engagement on system success: A meta-analytical integration of research findings , 1999, Inf. Manag..

[25]  Jane E. Klobas,et al.  User developed application success: sources and effects of involvement , 2008, Behav. Inf. Technol..

[26]  Lene Pries-Heje,et al.  Time, Attitude, and User Participation: How Prior Events Determine User Attitudes in ERP Implementation , 2008, Int. J. Enterp. Inf. Syst..

[27]  Alexander M. Maish.  A User's Behavior Toward His MIS , 1979, MIS Q..

[28]  Tore Dybå,et al.  Teaching evidence-based software engineering to university students , 2005, 11th IEEE International Software Metrics Symposium (METRICS'05).

[29]  Anders Edstrom,et al.  User Influence and the Success of MIS Projects: A Contingency Approach , 1977 .

[30]  Blake Ives,et al.  User Involvement in System Design: An Empirical Test of Alternative Approaches , 1981, Inf. Manag..

[31]  Gabriele Piccoli,et al.  Moving beyond user participation to achieve successful IS design , 2007, CACM.

[32]  Jay F. Nunamaker,et al.  Enabling the Effective Involvement of Multiple Users: Methods and Tools for Collaborative Software Engineering , 1997, J. Manag. Inf. Syst..

[33]  E. B. Swanson,et al.  Management Information Systems: Appreciation and Involvement , 1974 .

[34]  Robert Fildes,et al.  Influence of user participation on DSS use and decision accuracy , 2002 .

[35]  Walid Maalej,et al.  When users become collaborators: towards continuous and context-aware user input , 2009, OOPSLA Companion.

[36]  Kwasi Amoako-Gyampah,et al.  User involvement and user satisfaction: An exploratory contingency model , 1993, Inf. Manag..

[37]  Wan Adilah Wan Adnan,et al.  A survey on user involvement in software Development Life Cycle from practitioner's perspectives , 2010, 5th International Conference on Computer Sciences and Convergence Information Technology.

[38]  Bernd Pape,et al.  Using Groupware for Mediated Feedback , 2004 .

[39]  Tor Guimaraes,et al.  Client/Server System Success: Exploring the Human Side , 1997 .

[40]  Björn Regnell,et al.  Requirements are slipping through the gaps — A case study on causes & effects of communication gaps in large-scale software development , 2011, 2011 IEEE 19th International Requirements Engineering Conference.

[41]  David Gefen,et al.  Leadership and justice: Increasing non participating users' assessment of an IT through passive participation , 2008, Inf. Manag..

[42]  Kamel Rouibah,et al.  Effect of management support, training, and user involvement on system usage and satisfaction in Kuwait , 2009, Ind. Manag. Data Syst..

[43]  Leon A. Kappelman,et al.  The Respective Roles Of User Participation And User Involvement In Information System Implementation Success , 1991, ICIS.

[44]  Shivraj Kanungo,et al.  Understanding user participation and involvement in ERP use , 2000 .

[45]  Gert-Jan de Vreede,et al.  Collaborative Usability Testing to Facilitate Stakeholder Involvement , 2006, Value-Based Software Engineering.

[46]  Glen B. Alleman.  Agile Project Management Methods for ERP: How to Apply Agile Processes to Complex COTS Projects and Live to Tell about It , 2002, XP/Agile Universe.

[47]  James E. Hunton,et al.  Effects of the user participation process and task meaningfulness on key information system outcomes , 1997 .

[48]  Graham C. Low,et al.  Exploring Individual User Satisfaction Within User-Led Development , 1993, MIS Q..

[49]  Louis Raymond,et al.  Impact of task uncertainty, end-user involvement, and competence on the success of end-user computing , 1998, Inf. Manag..

[50]  BaronasAnn-Marie,et al.  Restoring a sense of control during implementation: how user involvement leads to system acceptance , 1988 .

[51]  Lars Witell,et al.  Collaboration with Customers - Understanding the Effect of Customer-Company Interaction in New Product Development , 2011, 2011 44th Hawaii International Conference on System Sciences.

[52]  Henri Barki,et al.  User participation and user involvement in information system development , 1991, Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences.

[53]  Henri Barki,et al.  Communication as a dimension of user participation , 2001 .

[54]  Trevor Wood-Harper,et al.  The finding of thorns: user participation in enterprise system implementation , 2002, DATB.

[55]  Gary Klein,et al.  User commitment and collaboration: Motivational antecedents and project performance , 2010, Inf. Softw. Technol..

[56]  Tiziana Catarci,et al.  A Three-Fold Integration Framework to Incorporate User-Centered Design into Agile Software Development , 2011, HCI.

[57]  B. Brabander,et al.  Successful Information System Development in Relation to Situational Factors Which Affect Effective Communication Between MIS-Users and EDP-Specialists , 1984 .

[58]  Steven L. Alter.  Development Patterns for Decision Support Systems , 1978, MIS Q..

[59]  Kristin Lofthus Hope,et al.  Configuring Designers? Using One Agile Project Management Methodology to Achieve User Participation , 2011 .

[60]  Volkmar Pipek,et al.  Fostering user-developer collaboration with infrastructure probes , 2008, CHASE.

[61]  Walid Maalej,et al.  On the Socialness of Software , 2011, 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing.

[62]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[63]  James J. Jiang,et al.  The Importance of Building a Foundation for User Involvement in Information System Projects , 2002 .

[64]  Mary Beth Rosson,et al.  Proceedings of the 15th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications , 2000, Conference on Object-Oriented Programming Systems, Languages, and Applications.

[65]  Mark A. Harris,et al.  A New Look at the Relationship between User Involvement in Systems Development and System Success , 2009, Communications of the Association for Information Systems.

[66]  C. R. Franz,et al.  Group process and conflict in system development , 1989 .

[67]  Tor Guimaraes,et al.  Empirically Testing Some Main User-Related Factors for Systems Development Quality , 2003, ICIQ.

[68]  Ramanath Subramanyam,et al.  User participation in software development projects , 2010, CACM.

[69]  Ulrike Abelein.  User-developer communication in large-scale IT projects , 2015 .

[70]  Michael Frese,et al.  Don't underestimate the problems of user centredness in software development projectsthere are many! , 1996, Behav. Inf. Technol..

[71]  James Noble,et al.  An Ideal Customer: A Grounded Theory of Requirements Elicitation, Communication and Acceptance on Agile Projects , 2010, Agile Software Development.

[72]  Stephen R. Hawk.  The Effects of User Involvement: Some Personality Determinants , 1993, Int. J. Man Mach. Stud..

[73]  Robert D. Macredie,et al.  The Importance of Context in Information System Design: An Assessment of Participatory Design , 1999, Requirements Engineering.

[74]  Yvonne Dittrich,et al.  How to Include Users in the Development of Off-the-Shelf Software: A Case for Complementing Participatory Design with Agile Development , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).

[75]  Jin Cheng,et al.  The Impact of Organizational Culture on ERP Assimilation: The Mediating Role of User Participation , 2010, 2010 2nd International Workshop on Database Technology and Applications.

[76]  Barbara Paech,et al.  A proposal for enhancing user-developer communication in large IT projects , 2012, 2012 5th International Workshop on Co-operative and Human Aspects of Software Engineering (CHASE).

[77]  AbeleinUlrike,et al.  Understanding the Influence of User Participation and Involvement on System Success --- a Systematic Mapping Study , 2015 .

[78]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[79]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[80]  Tor Guimaraes,et al.  The Relationship Between User Participation and User Satisfaction: An Investigation of Four Contingency Factors , 1994, MIS Q..

[81]  Pekka Abrahamsson,et al.  A case study on the impact of customer communication on defects in agile software development , 2006, AGILE 2006 (AGILE'06).

[82]  Tor Guimaraes,et al.  Empirically testing the outcomes of user involvement in DSS development , 1994 .

[83]  Li-Ling Hsu,et al.  Understanding the critical factors effect user satisfaction and impact of ERP through innovation of diffusion theory , 2008, Int. J. Technol. Manag..

[84]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[85]  Nathan Brewer,et al.  Improving communication between customers and developers , 2005, Agile Development Conference (ADC'05).

[86]  James E. Hunton,et al.  Involving Information System Users in Defining System Requirements: The Influence of Procedural Justice Perceptions on User Attitudes and Performance , 1996 .

[87]  George M. Marakas,et al.  The Impact of Operational User Participation on Perceived System Implementation Success: An Empirical Investigation , 2006, J. Comput. Inf. Syst..

[88]  Henry C. Lucas,et al.  Why information systems fail , 1975 .

[89]  Radhika Santhanam,et al.  An empirical investigation of ODSS impact on individuals and organizations , 2000, Decis. Support Syst..

[90]  Slinger Jansen,et al.  Customer Involvement in Requirements Management: Lessons from Mass Market Software Development , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[91]  Andrew Martin,et al.  A model of information systems development project performance , 2000, Inf. Syst. J..

[92]  Gary Klein,et al.  User involvement to enhance expertise in system development , 2008, Int. J. Internet Enterp. Manag..

[93]  Hilkka Merisalo-Rantanen,et al.  Engineering E-Collaboration Processes to Obtain Innovative End-User Feedback on Advanced Web-Based Information Systems , 2009, J. Assoc. Inf. Syst..

[94]  Jinjoo Lee,et al.  An exploratory contingency model of user participation and MIS use , 1986, Inf. Manag..

[95]  Barbara Begier.  Evolutionally Improved Quality of Intelligent Systems Following Their Users' Point of View , 2010, Advances in Intelligent Information and Database Systems.

[96]  James E. Tomayko,et al.  Human Aspects of Software Engineering: The Case of Extreme Programming , 2004, XP.

[97]  Naveed Saleem,et al.  An Empirical Test of the Contingency Approach to User Participation in Information Systems Development , 1996, J. Manag. Inf. Syst..

[98]  Kwasi Amoako-Gyampah,et al.  When Is User Involvement Not User Involvement , 1997 .

[99]  Angèle L. M. Cavaye,et al.  User participation in system development revisited , 1995, Inf. Manag..

[100]  Bruce Cheek,et al.  Does user involvement during information systems development improve data quality , 1998 .

[101]  Francisca Pérez,et al.  Towards the Involvement of End-Users within Model-Driven Development , 2011, IS-EUD.

[102]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[103]  Peter B. Seddon,et al.  A Partial Test and Development of Delone and Mclean's Model of IS Success , 1996, Australas. J. Inf. Syst..

[104]  W. Doll,et al.  The development of a tool for measuring the perceived impact of information technology on work , 1999 .

[105]  Henri Barki,et al.  User Participation, Conflict, and Conflict Resolution: The Mediating Roles of Influence , 1994, Inf. Syst. Res..

[106]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[107]  Shouqian Sun,et al.  Designing a Cooperative Learning System: A Scenario and Participatory Design Based Approach , 2008, 2008 International Symposium on Computational Intelligence and Design.

[108]  Gloria Mark,et al.  User Advocacy in Participatory Design: Designers' Experiences with a New Communication Channel , 2004, Computer Supported Cooperative Work (CSCW).

[109]  Alistair G. Sutcliffe,et al.  Tracing Requirements Errors to Problems in the Requirements Engineering Process , 1999, Requirements Engineering.

[110]  Cynthia Mathis Beath,et al.  The enactments and consequences of token, shared, and compliant participation in information systems development , 1996 .

[111]  Tor Guimaraes,et al.  Exploring expert system success factors for business process reengineering , 1998 .

[112]  Richard J. Boland,et al.  The Process and Product of System Design , 1978 .

[113]  Sari Kujala,et al.  User involvement: A review of the benefits and challenges , 2003, Behav. Inf. Technol..

[114]  Benjamin B. M. Shao,et al.  The relationship between user participation and system success: a simultaneous contingency approach , 2000, Inf. Manag..

[115]  William J. Doll,et al.  A Congruence Construct of User Involvement , 1991 .

[116]  Hsi-Peng Lu,et al.  The relationships between management styles, user participation, and system success over MIS growth stages , 1997, Inf. Manag..

[117]  Samuli Pekkola,et al.  Towards formalised end-user participation in information systems development process: bridging the gap between participatory design and ISD methodologies , 2006, PDC '06.

[118]  Sari Kujala,et al.  user involvement in product development by improving the analysis of user needs , 2016 .

[119]  David G. Hendry.  Public participation in proprietary software development through user roles and discourse , 2008, Int. J. Hum. Comput. Stud..

[120]  Ann-Marie K. Baronas,et al.  Restoring a Sense of Control During Implementation: How User Involvement Leads to System Acceptance , 1988, MIS Q..

[121]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[122]  Khaled El Emam,et al.  User participation in the requirements engineering process: An empirical study , 1996, Requirements Engineering.

[123]  Paula Kotzé,et al.  An overview of systems design and development methodologies with regard to the involvement of users and other stakeholders , 2003 .

[124]  Henri Barki,et al.  Delineating the dimensions of user participation: a replication and extension , 1997, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[125]  Samuli Pekkola,et al.  Reframing Humans in Information Systems Development , 2010 .

[126]  Marcus O'Connor,et al.  MIS success: why does it vary among users? , 1992, J. Inf. Technol..

[127]  Kwasi Amoako-Gyampah,et al.  Perceived usefulness, user involvement and behavioral intention: an empirical study of ERP implementation , 2007, Comput. Hum. Behav..

[128]  Shirley Gregor,et al.  User participation in decision support systems development: Influencing system outcomes , 2004, Eur. J. Inf. Syst..

[129]  Yvonne Dittrich,et al.  Agile Processes Enhancing User Participation for Small Providers of Off-the-Shelf Software , 2004, XP.

[130]  Rudy Hirschheim,et al.  User Experience with and Assessment of Participative Systems Design , 1985, MIS Q..

[131]  William J. Doll,et al.  The Collaborative Use of Information Technology: End-User Participation and Systems Success , 2001, Inf. Resour. Manag. J..

[132]  Dov Dori,et al.  A Software System Development Life Cycle Model for Improved Stakeholders' Communication and Collaboration , 2010, Int. J. Comput. Commun. Control.

[133]  Beatriz Sousa Santos,et al.  Using participatory design in a health information system , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[134]  Karlheinz Kautz,et al.  Investigating the design process: participatory design in agile software development , 2011, Inf. Technol. People.

[135]  Charles Russel Franz,et al.  Contingency factors affecting the user involvement role in the design of successful information systems. , 1979 .

[136]  Jong-min Choe,et al.  The Relationships among Performance of Accounting Information Systems, Influence Factors, and Evolution Level of Information Systems , 1996, J. Manag. Inf. Syst..

[137]  Mari-Ann Flyvholm,et al.  A study of the implementation process of an intervention to prevent work-related skin problems in wet-work occupations , 2006, International archives of occupational and environmental health.

[138]  Netta Iivari,et al.  Enculturation of user involvement in software development organizations - an interpretive case study in the product development context , 2004, NordiCHI '04.

[139]  Karlheinz Kautz,et al.  Customer and User Involvement in Agile Software Development , 2009, XP.

[140]  Jesper Simonsen,et al.  MUST: A Method for Participatory Design , 1998, Hum. Comput. Interact..

[141]  R. F. Powers,et al.  MisProject Management: Myths, Opinions, and Reality , 1973 .

[142]  Roy H. Igersheim,et al.  Managerial response to an information system , 1899, AFIPS '76.

[143]  Muneera Bano,et al.  User involvement in software development and system success: a systematic literature review , 2013, EASE '13.

[144]  Jorge J. Gómez-Sanz,et al.  Understanding the human context in requirements elicitation , 2010, Requirements Engineering.

[145]  Henri Barki,et al.  Explaining the Role of User Participation in Information System Use , 1994 .