The Role of User Participation in Information Systems Development: Implications from a Meta-Analysis

This study synthesizes the research findings of 82 empirical studies on user participation in information systems development (ISD). Various ISD outcomes are addressed using a classification scheme involving two broad categories-attitudinal/behavioral outcomes and productivity outcomes. The results demonstrate that user participation is minimally-to-moderately beneficial to ISD; its effects are comparatively stronger on attitudinal/behavioral outcomes than on productivity outcomes. This attitudinal/behavioral impact may largely be the result of the emphasis that has been placed on user participation by academics and consultants. The results of this analysis are compared to those of a meta-analysis in the broader management context of participation. The results are similar in terms of attitudinal outcomes, but different, and lesser, in terms of productivity outcomes. Since the current status of research in the broad area of participation is that the effects of participation are considered to be problematic, that status and the results of this study suggest that user participation alone may not be sufficient to achieve significantly improved ISD outcomes, and that different strategies should be employed based on the specific goals of ISD projects. If system acceptance is the ultimate goal, user participation should be designed to induce more psychological involvement among potential users. If productivity benefits are the focus, user participation should be designed to provide developers the needed domain knowledge. In sum, user participation should be treated as one of a number of means for ISD projects to be more successful.

[1]  M. Lynne Markus,et al.  Participation in Development and Implementation - Updating An Old, Tired Concept for Today's IS Contexts , 2004, J. Assoc. Inf. Syst..

[2]  Angèle L. M. Cavaye,et al.  User participation in system development revisited , 1995, Inf. Manag..

[3]  J. Thong.  Resource Constraints and Information Systems Implementation in Singaporean Small Businesses , 2001 .

[4]  Felix C. Brodbeck,et al.  Communication and performance in software development projects , 2001 .

[5]  Jong-Min Choe,et al.  The effects of user participation on the design of accounting information systems , 1998, Inf. Manag..

[6]  A. V. Sioukas,et al.  User involvement for effective customization: an empirical study on voice networks , 1995 .

[7]  Pascale Carayon,et al.  Sociotechnical issues in the implementation of imaging technology , 2000, Behav. Inf. Technol..

[8]  J. B. Ritchie,et al.  AN ANALYSIS OF QUANTITY AND QUALITY OF PARTICIPATION AS MEDIATING VARIABLES IN THE PARTICIPATIVE DECISION MAKING PROCESS , 1970 .

[9]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[10]  Marcus O'Connor,et al.  MIS success: why does it vary among users? , 1992, J. Inf. Technol..

[11]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[12]  Petter Gottschalk,et al.  Implementation of formal plans: the case of information technology strategy , 1999 .

[13]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[14]  Quey-Jen Yeh,et al.  Two conflict potentials during IS development , 2001, Inf. Manag..

[15]  Radhika Santhanam,et al.  An empirical investigation of ODSS impact on individuals and organizations , 2000, Decis. Support Syst..

[16]  Craig Standing,et al.  Do Project Manager's Utilise Potential Customers in E-Commerce Developments? , 2004 .

[17]  Paulus Insap Santosa,et al.  User involvement and user satisfaction with information-seeking activity , 2005, Eur. J. Inf. Syst..

[18]  Peter R. Monge,et al.  Participation, Satisfaction, and Productivity: A Meta-Analytic Review , 1986 .

[19]  Mark Keil,et al.  The user–developer communication process: a critical case study , 2003, Inf. Syst. J..

[20]  Jinjoo Lee,et al.  An exploratory contingency model of user participation and MIS use , 1986, Inf. Manag..

[21]  Tor Guimaraes,et al.  Successful Strategies for User Participation in Systems Development , 1997, J. Manag. Inf. Syst..

[22]  Tor Guimaraes,et al.  Factors important to expert systems success a field test , 1996, Inf. Manag..

[23]  Naveed Saleem,et al.  An Empirical Test of the Contingency Approach to User Participation in Information Systems Development , 1996, J. Manag. Inf. Syst..

[24]  C. R. Franz,et al.  ORGANIZATIONAL CONTEXT, USER INVOLVEMENT, AND THE USEFULNESS OF INFORMATION SYSTEMS* , 1986 .

[25]  Cynthia M. Jackson,et al.  Toward an Understanding of the Behavioral Intention to Use an Information System , 1997 .

[26]  Robert Fildes,et al.  Influence of user participation on DSS use and decision accuracy , 2002 .

[27]  Sid L. Huff,et al.  Implementing Decision Support Systems: Correlates Of User Satisfaction And System Usage , 1990 .

[28]  June M. Verner,et al.  Quantitative models for early prediction of software development success: a practitioner's perspective , 2002 .

[29]  William R. King,et al.  The Design of Management Information Systems an Information Analysis Approach , 1975 .

[30]  Daniel Robey,et al.  User Attitudes and Management Information System Use , 1979 .

[31]  Henri Barki,et al.  Explaining the Role of User Participation in Information System Use , 1994 .

[32]  Yuh-Min Wang,et al.  Development of a Tool for Measuring Key-User Satisfaction in an ERP Environment , 2002 .

[33]  A. Field.  Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods. , 2001, Psychological methods.

[34]  Kevin James Pyle,et al.  AN EMPIRICAL EXAMINATION OF CRITICAL SUCCESS FACTORS FOR USER BASED IMPLEMENTATION PROJECTS IN MANAGEMENT INFORMATION SYSTEMS , 1986 .

[35]  James E. Hunton,et al.  Involving Information System Users in Defining System Requirements: The Influence of Procedural Justice Perceptions on User Attitudes and Performance , 1996 .

[36]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[37]  George M. Marakas,et al.  The Impact of Operational User Participation on Perceived System Implementation Success: An Empirical Investigation , 2006, J. Comput. Inf. Syst..

[38]  Donald L. Amoroso,et al.  Organizational issues of end-user computing , 1988, DATB.

[39]  Tor Guimaraes,et al.  Empirically testing ES success factors in business process reengineering , 1997 .

[40]  Erran Carmel,et al.  PD and joint application design: a transatlantic comparison , 1993, CACM.

[41]  William R. King,et al.  Manager-analyst teamwork in MISCooperation vital in systems design , 1971 .

[42]  Andrew Martin,et al.  A model of information systems development project performance , 2000, Inf. Syst. J..

[43]  Edwin A. Locke,et al.  Cognitive and motivational frameworks in U.S. research on participation: A meta-analysis of primary effects , 1997 .

[44]  E. A. Locke,et al.  Participation in Decision-Making: One More Look , 1979 .

[45]  Paul Beynon-Davies,et al.  ‘It’s lots of bits of paper and ticks and post‐it notes and things . . .’: a case study of a rapid application development project , 2000, Inf. Syst. J..

[46]  James E. Hunton,et al.  Effects of User Participation in Systems Development: A Longitudinal Field Experiment , 1997, MIS Q..

[47]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[48]  Blake Ives,et al.  User Involvement in System Design: An Empirical Test of Alternative Approaches , 1981, Inf. Manag..

[49]  Richard P. Vlosky,et al.  Impact of corporate orientation of information technology adoption in the United States forest products industry , 2003 .

[50]  Phyllisis Mei-Mei Ngin,et al.  Organizational analyses of computer user acceptance among nurses , 1993 .

[51]  Weidong Xia,et al.  Grasping the complexity of IS development projects , 2004, CACM.

[52]  Terry L. Fox.  What Makes a System Development Project Successful? New Answers from a Project Manager's Perspective , 2006, Americas Conference on Information Systems.

[53]  Tor Guimaraes,et al.  Empirically testing the outcomes of user involvement in DSS development , 1994 .

[54]  John E. Hunter,et al.  Comparison of three meta-analysis methods revisited : An analysis of Johnson, Mullen, and Salas (1995) , 1999 .

[55]  Dorothy Leonard-Barton,et al.  Developer-User Interaction and User Satisfaction in Internal Technology Transfer , 1993 .

[56]  Daniel Robey,et al.  Perceptions of Conflict and Success in Information Systems Development Projects , 1993, J. Manag. Inf. Syst..

[57]  F. W. Mcfarlan,et al.  The information Archipelago ― Governing the new world , 1984 .

[58]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[59]  S. Thomas Foster,et al.  On the differences between information systems' users and analysts: Managing perceptions of systems quality , 1998 .

[60]  Elisa Fredericks,et al.  Cross‐functional involvement in new product development , 2005 .

[61]  Jong-min Choe,et al.  The Relationships among Performance of Accounting Information Systems, Influence Factors, and Evolution Level of Information Systems , 1996, J. Manag. Inf. Syst..

[62]  Tor Guimaraes,et al.  Assessing Expert Systems Impact on Users' Jobs , 1995, J. Manag. Inf. Syst..

[63]  William J. Doll,et al.  A Congruence Construct of User Involvement , 1991 .

[64]  Hsi-Peng Lu,et al.  The relationships between management styles, user participation, and system success over MIS growth stages , 1997, Inf. Manag..

[65]  Jr. Henry C. Lucas,et al.  Performance and the Use of an Information System , 1975 .

[66]  D. Vollrath,et al.  Employee Participation: Diverse Forms and Different Outcomes , 1988 .

[67]  Peter Johnson,et al.  Costs and Benefits of User Involvement in Design: Practitioners' Views , 1996, BCS HCI.

[68]  S. T. Foster,et al.  User involvement during information systems development: a comparison of analyst and user perceptions of system acceptance , 1999 .

[69]  Gregg B. Jackson,et al.  Meta-Analysis: Cumulating Research Findings Across Studies , 1982 .

[70]  William R. King,et al.  Understanding the Role and Methods of Meta-Analysis in IS Research , 2005, Commun. Assoc. Inf. Syst..

[71]  James E. Hunton,et al.  Effects of the user participation process and task meaningfulness on key information system outcomes , 1997 .

[72]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[73]  Tom Butler,et al.  The Relationship Between User Participation and the Management of Change Surrounding the Development of Information Systems: A European Perspective , 2001, J. Organ. End User Comput..

[74]  Mark I. Hwang,et al.  The effect of user engagement on system success: A meta-analytical integration of research findings , 1999, Inf. Manag..

[75]  Bruce Cheek,et al.  Does user involvement during information systems development improve data quality , 1998 .

[76]  Jay Aronson,et al.  An empirical study of the relationship between user participation and user satisfaction for expert systems development in the information technology services industry , 2000 .

[77]  Anders Edstrom,et al.  User Influence and the Success of MIS Projects: A Contingency Approach , 1977 .

[78]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[79]  James A. Conway,et al.  The Myth, Mystery, and Mastery of Participative Decision Making in Education , 1984 .

[80]  William R. King,et al.  Note-Participative Design of Strategic Decision Support Systems: An Empirical Assessment , 1981 .

[81]  Donald A. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[82]  Joseph Bradley,et al.  ERP Training and User Satisfaction , 2004, AMCIS.

[83]  F. McFarlan,et al.  The information archipelago--plotting a course. , 1983, Harvard business review.

[84]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[85]  William R. King,et al.  A meta-analysis of the technology acceptance model , 2006, Inf. Manag..

[86]  E. B. Swanson,et al.  Management Information Systems: Appreciation and Involvement , 1974 .

[87]  John R. Weitzel.  Managing the implementation of information systems : a multiple industry investigation of process, information systems-user relationships and project outcomes , 1985 .

[88]  James E. Hunton,et al.  User Participation in Defining System Interface Requirements: An Issue of Procedural Justice , 1997 .

[89]  Ramaraj Palanisamy,et al.  Strategic information systems planning model for building flexibility and success , 2005, Ind. Manag. Data Syst..

[90]  Cynthia Mathis Beath,et al.  The enactments and consequences of token, shared, and compliant participation in information systems development , 1996 .

[91]  Richard J. Boland,et al.  The Process and Product of System Design , 1978 .

[92]  Sari Kujala,et al.  User involvement: A review of the benefits and challenges , 2003, Behav. Inf. Technol..

[93]  Susan Ciccantelli,et al.  From experience: Consumer idealized design: Involving consumers in the product development process , 1993 .

[94]  Benjamin B. M. Shao,et al.  The relationship between user participation and system success: a simultaneous contingency approach , 2000, Inf. Manag..

[95]  Kwasi Amoako-Gyampah,et al.  User involvement and user satisfaction: An exploratory contingency model , 1993, Inf. Manag..

[96]  Leela Damodaran,et al.  User involvement in the systems design process-a practical guide for users , 1996, Behav. Inf. Technol..

[97]  E. Vanlommel,et al.  The Organization of Electronic Data Processing (EDP) Activities and Computer Use , 1975 .

[98]  Michael Frese,et al.  Don't underestimate the problems of user centredness in software development projectsthere are many! , 1996, Behav. Inf. Technol..

[99]  Scott Isensee,et al.  User-Centered Design: An Integrated Approach with Cdrom , 2001 .

[100]  Barbara Dastoor,et al.  An empirical study of the organizational benefits from implementing end user computing applications: its relevance to user participation , 2001 .

[101]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[102]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[103]  Edwin A. Locke,et al.  Fact and Fiction in Analyzing Research on Participative Decision Making: A Critique of Cotton, Vollrath, Froggatt, Lengnick-Hall, and Jennings , 1990 .

[104]  Paul John Steinbart,et al.  The Effects of Explanation Type and User Involvement on Learning from and Satisfaction with Expert Systems , 1998 .

[105]  Edwin A. Locke,et al.  Cognitive and motivational effects of participation: A mediator study , 1994 .

[106]  Warren Allan Beatty,et al.  Determining factors of microcomputer-based management information system user satisfaction , 1986 .

[107]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[108]  Blake Ives,et al.  Executive Involvement and Participation in the Management of Information Technology , 1991, MIS Q..

[109]  Leon A. Kappelman,et al.  The Respective Roles Of User Participation And User Involvement In Information System Implementation Success , 1991, ICIS.

[110]  Kalle Lyytinen,et al.  User participation in knowledge update of expert systems , 1997, Inf. Manag..

[111]  Daniel Robey,et al.  The organizational and cultural context of systems implementation: Case experience from Latin America , 1989, Inf. Manag..

[112]  Charles R. Franz,et al.  User Leadership in the Systems Development Life Cycle: A Contingency Model , 1985, J. Manag. Inf. Syst..

[113]  Anand Vadapalli,et al.  Information technology project outcomes: user participation structures and the impact of organization behavior and human resource management issues , 2000 .

[114]  Tsang-Hsiung Lee,et al.  The Effects Of User Participation On System Success: Toward A Contingency Theory Of User Satisfaction , 1991, ICIS.

[115]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[116]  Kenneth L. Kraemer,et al.  Institutional Factors in Information Technology Innovation , 1994, Inf. Syst. Res..

[117]  Barbara Wixom,et al.  An Empirical Investigation of the Factors Affecting Data Warehousing Success , 2001, MIS Q..

[118]  Ralph D. Loftin,et al.  SIM Competition Paper: Organization Development Methods in the Management of the Information Systems Function , 1982, MIS Q..

[119]  R. Rosenthal,et al.  Meta-analysis: recent developments in quantitative methods for literature reviews. , 2001, Annual review of psychology.

[120]  Alexander M. Maish.  A User's Behavior Toward His MIS , 1979, MIS Q..

[121]  Bill C. Hardgrave,et al.  Toward a Contingency Model for Selecting an Information System Prototyping Strategy , 1999, J. Manag. Inf. Syst..

[122]  John E. Hunter,et al.  Comparison of Three Meta-Analysis Methods Revisited: An Analysis of Johnson, Mullen, and Salas (1995) , 1999 .

[123]  Thomas D. Cook,et al.  Threats to the validity of research syntheses , 1994 .

[124]  Suzanne Rivard,et al.  An Integrative Contingency Model of Software Project Risk Management , 2001, J. Manag. Inf. Syst..

[125]  Rudy Hirschheim,et al.  User Experience with and Assessment of Participative Systems Design , 1985, MIS Q..

[126]  John A. Wagner,et al.  Shared influence and organizational behavior: A meta-analysis of situational variables expected to moderate participation-outcome relationships. , 1987 .

[127]  William J. Doll,et al.  The Collaborative Use of Information Technology: End-User Participation and Systems Success , 2001, Inf. Resour. Manag. J..

[128]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[129]  Blair T. Johnson,et al.  Comparison of three major meta-analytic approaches. , 1995 .

[130]  Kenneth A. Kozar,et al.  A User Generated Information System: An Innovative Development Approach , 1987, MIS Q..

[131]  David Graham Wastell,et al.  An information system profile of the UK manufacturing sector , 1995, J. Inf. Technol..

[132]  K. Eisenhardt,et al.  Accelerating Adaptive Processes: Product Innovation in the Global Computer Industry , 1995 .

[133]  Tor Guimaraes,et al.  The Relationship Between User Participation and User Satisfaction: An Investigation of Four Contingency Factors , 1994, MIS Q..

[134]  Michael J. Ginzberg,et al.  Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered Questions , 1981 .

[135]  Rudy Hirschheim,et al.  Assessing participative systems design: Some conclusions from an exploratory study , 1983, Inf. Manag..

[136]  Stephen R. Hawk,et al.  Successful system development: the effect of situational factors on alternative user roles , 1991 .

[137]  Debra Howcroft,et al.  Paradoxes of participatory practices: the Janus role of the systems developer , 2003, Inf. Organ..

[138]  John A. Wagner,et al.  Participation's Effects On Performance and Satisfaction: A Reconsideration Of Research Evidence , 1994 .

[139]  Adel M. Aladwani.  IT Planning Effectiveness in a Developing Country , 2001 .

[140]  Michael J. Ginzberg,et al.  Key Recurrent Issues in the MIS Implementation Process , 1981, MIS Q..

[141]  Anders Edstrom,et al.  Successful Information System Development Projects , 1977 .

[142]  Henri Barki,et al.  Rethinking the Concept of User Involvement , 1989, MIS Q..