Enterprise Systems Success: A Measurement Model

This paper presents a validated measurement model and instrument for assessing enterprise systems success from multiple perspectives. The final validated study model employs 27 measures of the four dimensions: information quality, system quality, individual impact, and organizational impact. The model is empirically tested with survey data gathered from 27 public sector organizations that implemented SAP R/3 in the late 1990s. The study consists of an exploratory inventory survey (model building) to identify the salient success dimensions and measures, followed by a confirmatory weights survey, for testing model validity (model testing). Test results demonstrate the discriminant validity of the four dimensions, as well as their convergence on a single higher-order phenomenon: enterprise systems success (ESS). Criterion validity testing further demonstrates the additivity of the four dimensions of success, and the completeness of the resultant overarching, second-order measure of ESS.

[1]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[2]  Peter G. W. Keen,et al.  Mis Research: Reference disciplines and a Cumulative Tradition , 1980, ICIS.

[3]  Charles H. Kriebel,et al.  Information Technologies and Business Value: An Analytic and Empirical Investigation , 1995, Inf. Syst. Res..

[4]  She-I Chang,et al.  Major Issues with Enterprise Systems: A Case Study and Survey of Five Government Agencies , 2003 .

[5]  Wanda J. Orlikowski,et al.  A Short Form Measure of User Information Satisfaction: A Psychometric Evaluation and Notes on Use , 1987, J. Manag. Inf. Syst..

[6]  Margi Levy,et al.  Developing a 3-D model of information systems success , 1998 .

[7]  Bill Lehr,et al.  Information technology and its impact on firm-level productivity: evidence from government and private data sources, 1977-1993 , 1999 .

[8]  Chee-Sing Yap,et al.  Information Systems Effectiveness: A User Satisfaction Approach , 1996, Inf. Process. Manag..

[9]  S. Winter.  Knowledge and Competence as Strategic Assets , 1987 .

[10]  Joan Antoni Pastor,et al.  Enterprise Resource Planning Systems Research: An Annotated Bibliography , 2001, Commun. Assoc. Inf. Syst..

[11]  Ali F. Farhoomand,et al.  A structural model of end user computing satisfaction and user performance , 1996, Inf. Manag..

[12]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[13]  Kuldeep Kumar,et al.  Post implementation evaluation of computer-based information systems: current practices , 1990, Commun. ACM.

[14]  Daniel Robey,et al.  User Attitudes and Management Information System Use , 1979 .

[15]  John S. Chandler.  A Multiple Criteria Approach for Evaluating Information Systems , 1982, MIS Q..

[16]  Richard O. Mason,et al.  Measuring information output: A communication systems approach , 1978, Inf. Manag..

[17]  Tor Guimaraes,et al.  Assessing Expert Systems Impact on Users' Jobs , 1995, J. Manag. Inf. Syst..

[18]  Peter M. Bentler,et al.  Practical Issues in Structural Modeling , 1987 .

[19]  Albert H. Segars,et al.  Re-examining perceived ease of use and usefulness , 1993 .

[20]  Darshana Sedera,et al.  Measuring enterprise systems success: the importance of a multiple stakeholder perspective , 2004, ECIS.

[21]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[22]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[23]  James C. Anderson,et al.  STRUCTURAL EQUATION MODELING IN PRACTICE: A REVIEW AND RECOMMENDED TWO-STEP APPROACH , 1988 .

[24]  G. Hofstede.  Culture′s Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations , 2001 .

[25]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[26]  Guy G. Gable,et al.  A multidimensional model of client success when engaging external consultants , 1996 .

[27]  Joyce J. Elam,et al.  Executive Information Systems: Their Impact on Executive Decision Making , 1994, J. Manag. Inf. Syst..

[28]  Peter B. Seddon.  A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[29]  Jen-Her Wu,et al.  An examination of ERP user satisfaction in Taiwan , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[30]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[31]  N. Melone.  A theoretical assessment of the user-satisfaction construct in information systems research , 1990 .

[32]  Henri Barki,et al.  Change, attitude to change, and decision support system success , 1985, Inf. Manag..

[33]  K. Cameron,et al.  Organizational paradox and transformation. , 1988 .

[34]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[35]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[36]  S ChandlerJohn.  A multiple criteria approach for evaluating information systems , 1982 .

[37]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[38]  A. Barua,et al.  The Information Technology Productivity Paradox Revisited: A Theoretical and Empirical Investigation in the Manufacturing Sector , 1997 .

[39]  Richard P. Bagozzi,et al.  "Issues in the application of covariance structure analysis": A further comment. , 1983 .

[40]  Maarten Gelderman,et al.  The relation between user satisfaction, usage of information systems and performance , 1998, Inf. Manag..

[41]  Jaak Jurison,et al.  The temporal nature of IS benefits: A longitudinal study , 1996, Inf. Manag..

[42]  Erik Brynjolfsson,et al.  The productivity paradox of information technology , 1993, CACM.

[43]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[44]  H. Marsh,et al.  Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. , 1985 .

[45]  Peter A. Todd,et al.  Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication , 1992, MIS Q..

[46]  Detmar W. Straub,et al.  Structural Equation Modeling and Regression: Guidelines for Research Practice , 2000, Commun. Assoc. Inf. Syst..

[47]  Peter B. Seddon,et al.  A Comprehensive Framework for Classifying the Benefits of ERP Systems , 2000 .

[48]  Scott T. Young,et al.  Hospital Materials Management: Systems and Performance , 1989 .

[49]  Jr. Henry C. Lucas,et al.  Performance and the Use of an Information System , 1975 .

[50]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[51]  G. Gable,et al.  Enterprise Resource Planning Systems Impacts : a Delphi study of Australian Public Sector Organizations , 2002 .

[52]  Kristi Yuthas,et al.  Material matters: Assessing the effectiveness of materials management IS , 1998, Inf. Manag..

[53]  Michael J. Ginzberg,et al.  Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered Questions , 1981 .

[54]  John Rohrbaugh,et al.  A Spatial Model of Effectiveness Criteria: Towards a Competing Values Approach to Organizational Analysis , 1983 .

[55]  D. Whetten,et al.  Organizational Effectiveness: A Comparison of Multiple Models , 1984 .

[56]  E. Brynjolfsson,et al.  Paradox Lost? Firm-Level Evidence on the Returns to Information Systems Spending , 1996 .

[57]  Benn R. Konsynski,et al.  An Examination of the Interaction between Technology, Methodology and Information Systems: a tripartite View · , 1980, ICIS.

[58]  Michael H. Zack,et al.  Rethinking the Knowledge-Based Organization , 2003 .

[59]  K. Wiig.  Knowledge management: Where did it come from and where will it go? , 1997 .

[60]  Petri Hallikainen,et al.  Reasons for ERP Acquisition , 2004, ICEIS.

[61]  John Leslie King,et al.  Cost-Benefit Analysis in Information Systems Development and Operation , 1978, CSUR.

[62]  Michael Rosemann,et al.  Using performance measurement models for benefit realization with enterprise systems the Queensland government approach , 2001, ECIS.

[63]  James C. Wetherbe,et al.  Key Information Systems Issues for the 1980's , 1984, MIS Q..

[64]  D. Turbide,et al.  WHAT IS ERP , 1999 .

[65]  Diane Wilson.  Assessing the impact of information technology on organizational performance , 1993 .

[66]  Michael Ishman,et al.  Measuring Information Success at the Individual Level in Cross-Cultural Environments , 1996 .

[67]  Marja Nissinen,et al.  The Baltics as a business location for information technology and electronics industries , 2002 .

[68]  Wynne W. Chin,et al.  Structural equation modeling analysis with small samples using partial least squares , 1999 .

[69]  Paul Attewell,et al.  Computing and organizations: what we know and what we don't know , 1984, CACM.

[70]  M. Morton,et al.  The corporation of the 1990s: Information technology and organizational transformation , 1993 .

[71]  Peter B. Seddon,et al.  A Partial Test and Development of Delone and Mclean's Model of IS Success , 1996, Australas. J. Inf. Syst..

[72]  E. B. Andersen,et al.  Modern factor analysis , 1961 .

[73]  Henk Akkermans,et al.  Vicious and virtuous cycles in ERP implementation: a case study of interrelations between critical success factors , 2002, Eur. J. Inf. Syst..

[74]  Michael Rosemann,et al.  Australasian ( ACIS ) 2001 A Balanced Scorecard Approach to Enterprise Systems Performance Measurement , 2001 .

[75]  Kenneth L. Kraemer,et al.  Executives’ Perceptions of the Business Value of Information Technology: A Process-Oriented Approach , 2000, J. Manag. Inf. Syst..

[76]  Thompson S. H. Teo,et al.  An empirical study of the performance impact of computerization in the retail industry , 1998 .

[77]  Wynne W. Chin.  Issues and Opinion on Structural Equation Modeling by , 2009 .

[78]  Richard Harris,et al.  SMIS Members: A Membership Analysis , 1982, MIS Q..

[79]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[80]  Ilze Zigurs,et al.  A Theory of Task/Technology Fit and Group Support Systems Effectiveness , 1998, MIS Q..

[81]  S. Miller,et al.  Productivity growth in large US commercial banks: The initial post-deregulation experience , 2001 .

[82]  Erik Brynjolfsson,et al.  Information Technology and Productivity: A Review of the Literature , 1996, Adv. Comput..

[83]  Henri Barki,et al.  Rethinking the Concept of User Involvement , 1989, MIS Q..

[84]  Ali H. Mansour,et al.  An empirical investigation of the determinants of computer-based information system performance , 1980 .

[85]  Magid Igbaria,et al.  Correlates of user satisfaction with end user computing: An exploratory study , 1990, Inf. Manag..

[86]  Gerald L. Matlin,et al.  SIM Competition Paper: What Is the Value of Investment in Information Systems? , 1979, MIS Q..

[87]  Kenneth D. Mackenzie,et al.  Paradigm Development in the Social Sciences: A Proposed Research Strategy , 1978 .

[88]  Darshana Sedera,et al.  ERP success: Does organisation Size Matter? , 2003, PACIS.

[89]  James C. Wetherbe,et al.  Key issues in information systems management , 1987 .

[90]  M. Al-Mashari.  Enterprise resource planning (ERP) systems: a research agenda , 2002 .

[91]  Carol Stoak Saunders,et al.  Measuring Performance of the Information Systems Function , 1992, J. Manag. Inf. Syst..

[92]  Leon A. Kappelman,et al.  A Comprehensive Model for Assessing the Quality and Productivity of the Information Systems Function: Toward a Theory for Information Systems Assessment , 1997 .

[93]  L MyersBarry,et al.  A Comprehensive Model for Assessing the Quality and Productivity of the Information Systems Function , 1997 .

[94]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[95]  P. Ein-Dor,et al.  Organizational Context and the Success of Management Information Systems , 1978 .

[96]  Ingoo Han,et al.  Performance measure of information systems (IS) in evolving computing environments: an empirical investigation , 2003, Inf. Manag..

[97]  Amy W. Gatian.  Is user satisfaction a valid measure of system effectiveness? , 1994, Inf. Manag..

[98]  Peter B. Seddon,et al.  A Partial Test and Development of the DeLone and McLean Model of IS Success , 1994, ICIS.

[99]  J OrlikowskiWanda,et al.  A short-form measure of user information satisfaction: a psychometric evaluation and notes on use , 1988 .

[100]  K. Wiig.  Integrating intellectual capital and knowledge management , 1997 .

[101]  Margaret Tan,et al.  The consequences of information technology acceptance on subsequent individual performance , 1997, Inf. Manag..

[102]  Andrew Martin,et al.  The 3-D Model of Information Systems Success: The Search for the Dependent Variable Continues , 1996 .

[103]  M. Markus,et al.  The Enterprise System Experience— From Adoption to Success , 2000 .

[104]  Sr Hawk,et al.  Measurement biases in user involvement research , 1990 .

[105]  Subhasish Dasgupta,et al.  MEASUREMENT OF USER SATISFACTION WITH WEB-BASED INFORMATION SYSTEMS: AN EMPIRICAL STUDY , 2002 .

[106]  Timo Saarinen,et al.  An expanded instrument for evaluating information system success , 1996, Inf. Manag..

[107]  Darshana Sedera,et al.  Knowledge Management for ERP success , 2003, PACIS.