Information systems effectiveness: The construct space and patters of application

Abstract Measuring the specific contribution(s) of Information Systems (IS) to organizational effectiveness remains a critical concern of both academic and practitioner communities. Reflecting the importance of this issue, over the past decade, a large number of performance criteria have been developed and empirically tested. This growing array of measurement criteria provides a foundation for defining the construct space of IS effectiveness and prescribing criteria for specific managerial needs. Although important research has been started in this area, additional work is needed to define the domain of IS effectiveness further and provide insight into the context within which various classes of performance criteria are used. Through meta-analysis of leading IS journals, this study identifies six classes of effectiveness measures that are then mapped onto various evaluative contexts. The construct space and evaluative context developed in this study provide a new way of conceptualizing and applying the many evaluative criteria associated with IS.

[1]  Lee L. Gremillion,et al.  Justifying Decision Support and Office Automation Systems , 1985, J. Manag. Inf. Syst..

[2]  Donald H. Bender,et al.  Financial Impact of Information Processing , 1986, J. Manag. Inf. Syst..

[3]  William H. DeLone.  Determinants of Success for Computer Usage in Small Business , 1988, MIS Q..

[4]  Ali R. Montazemi,et al.  Factors Affecting Information Satisfaction in the Context of the Small Business Environment , 1988, MIS Q..

[5]  Rajiv Sabherwal,et al.  The Alignment between Organizational Critical Success Factors and Information Technology Capability in Academic Institutions , 1994 .

[6]  P. Keen.  Value Analysis: Justifying Decision Support Systems , 1993 .

[7]  Scott Hamilton,et al.  The journal communication system for MIS research , 1982, DATB.

[8]  Kyung Kyu Kim,et al.  Structure and Perceived Effectiveness of Software Development Subunits: A Task Contingency Analysis , 1992, J. Manag. Inf. Syst..

[9]  Ronald E. Rice,et al.  Relating Electronic Mail Use and Network Structure to R&D Work Networks and Performance , 1994, J. Manag. Inf. Syst..

[10]  Peter Weill,et al.  An Assessment of the Contingency Theory of Management Information Systems , 1989, J. Manag. Inf. Syst..

[11]  Louis Raymond,et al.  Organizational Context and Information Systems Success: A Contingency Approach , 1990, J. Manag. Inf. Syst..

[12]  Marion G. Sobol,et al.  The relationship between computerization and performance: A strategy for maximizing the economic benefits of computerization , 1983, Inf. Manag..

[13]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[14]  Bernadette Szajna,et al.  Software Evaluation and Choice: Predictive Validation of the Technology Acceptance Instrument , 1994, MIS Q..

[15]  Dennis F. Galletta,et al.  Some Cautions on the Measurement of User Information Satisfaction , 1989 .

[16]  Kim S. Cameron,et al.  Measuring Organizational Effectiveness in Institutions of Higher Education. , 1978 .

[17]  Jinjoo Lee,et al.  An exploratory contingency model of user participation and MIS use , 1986, Inf. Manag..

[18]  C. R. Franz,et al.  ORGANIZATIONAL CONTEXT, USER INVOLVEMENT, AND THE USEFULNESS OF INFORMATION SYSTEMS* , 1986 .

[19]  Scott Hamilton,et al.  Evaluating Information Systems Effectiveness - Part II: Comparing Evaluator Viewpoints , 1981, MIS Q..

[20]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[21]  Paul H. Cheney,et al.  Concepts, Theory, and Techniques: FACTORS AFFECTING THE PERCEIVED UTILIZATION OF COMPUTER-BASED DECISION SUPPORT SYSTEMS IN THE OIL INDUSTRY , 1982 .

[22]  J. Miller,et al.  Measuring the Effectiveness of Computer-Based Information Systems in the Financial Services Sector , 1987, MIS Q..

[23]  F. McFarlan,et al.  Corporate Information Systems Management: Issues Facing Senior Executives , 1995 .

[24]  François Bergeron,et al.  The Use of Critical Success Factors in Evaluation of Information Systems: A Case Study , 1989, J. Manag. Inf. Syst..

[25]  William R. King,et al.  Impacts of End-User and Information Center Characteristics on End-User Computing Support , 1994, J. Manag. Inf. Syst..

[26]  William R. King,et al.  The Development of a Measure for End-User Computing Support , 1994 .

[27]  Robert P. Bostrom,et al.  A Longitudinal Study of the Impact of Group Decision Support Systems on Group Development , 1990, J. Manag. Inf. Syst..

[28]  Henri Barki,et al.  Change, attitude to change, and decision support system success , 1985, Inf. Manag..

[29]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[30]  Richard M. Steers.  Problems in the Measurement of Organizational Effectiveness , 1975 .

[31]  Michael R. Vitale,et al.  Creating Competitive Advantage with Interorganizational Information Systems , 1988, MIS Q..

[32]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[33]  P. Ein-Dor,et al.  Organizational Context and the Success of Management Information Systems , 1978 .

[34]  Garry D. Brewer,et al.  Assessing Outcomes and Effects , 1983 .

[35]  Michael R. Vitale,et al.  The Growing Risks of Information Systems Success , 1986, MIS Q..

[36]  R. Kelly Rainer,et al.  Toward Development of the End User Computing Construct in a University Setting , 1993 .

[37]  Peter G. W. Keen,et al.  Invited Article: Value Analysis: Justifying Decision Support Systems , 1981, MIS Q..

[38]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[39]  Peter B. Seddon,et al.  A Partial Test and Development of Delone and Mclean's Model of IS Success , 1996, Australas. J. Inf. Syst..

[40]  D. Whetten,et al.  12 – Some Conclusions about Organizational Effectiveness , 1983 .

[41]  Sree Nilakanta,et al.  Measurement: A blueprint for theory-building in MIS , 1994, Inf. Manag..

[42]  John J. Sviokla.  A field-based investigation into the effect of expert system use on the information processing capacity of the firm , 1989, ICIS '89.

[43]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[44]  Graham C. Low,et al.  Exploring Individual User Satisfaction Within User-Led Development , 1993, MIS Q..

[45]  Dan R. Pieptea,et al.  Price and Value of Decision Support Systems , 1987, MIS Q..

[46]  William J. Doll,et al.  A Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument , 1994, MIS Q..

[47]  Blake Ives,et al.  Executive Involvement and Participation in the Management of Information Technology , 1991, MIS Q..

[48]  Jane M. Howell,et al.  Influence of Experience on Personal Computer Utilization: Testing a Conceptual Model , 1994, J. Manag. Inf. Syst..

[49]  Tor Guimaraes,et al.  Antecedents and Consequences of Job Satisfaction among Information Center Employees , 1993, J. Manag. Inf. Syst..

[50]  Gerald L. Matlin,et al.  SIM Competition Paper: What Is the Value of Investment in Information Systems? , 1979, MIS Q..

[51]  P. J. S. Bruwer,et al.  A descriptive model of success for computer-based information systems , 1984, Inf. Manag..

[52]  Izak Benbasat,et al.  Cost/Benefit Analysis of Computer Based Message Systems , 1983, MIS Q..

[53]  Arthur G. Money,et al.  The Quantification of Decision Support Benefits Within the Context of Value Analysis , 1988, MIS Q..

[54]  David A. Whetten,et al.  1 – Organizational Effectiveness: One Model or Several? , 1983 .

[55]  Wynne W. Chin,et al.  Applying Adaptive Structuration Theory to Investigate the Process of Group Support Systems Use , 1992, J. Manag. Inf. Syst..

[56]  Ken Peffers,et al.  The Impact of Information Technology Investment Announcements on the Market Value of the Firm , 1993, Inf. Syst. Res..

[57]  Richard T. Watson,et al.  Service Quality: A Measure of Information System Effectiveness , 1995, MIS Q..

[58]  Jr. Raymond McLeod.  Management information systems: A study of computer-based information systems , 1993 .

[59]  Suzanne Rivard,et al.  ICIS Paper: User Developed Applications: Evaluation of Success from the DP , 1984, MIS Q..

[60]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[61]  Varun Grover,et al.  Strategic Information Systems Revisited: A Study in Sustainability and Performance , 1994, MIS Q..

[62]  Scott Hamilton,et al.  MIS research strategies , 1982, Inf. Manag..

[63]  Ramesh Sharda,et al.  Decision support system effectiveness: a review and an empirical test , 1988 .

[64]  Kyung Kyu Kim,et al.  Task characteristics, decentralization, and the success of hospital information systems , 1990, Inf. Manag..

[65]  Timothy Paul Cronan,et al.  On the Test-Retest Reliability of Perceived Usefulness and Perceived Ease of Use Scales , 1993 .

[66]  John Rohrbaugh,et al.  A Spatial Model of Effectiveness Criteria: Towards a Competing Values Approach to Organizational Analysis , 1983 .

[67]  Jon Hartwick,et al.  The Impact of Automated Office Systems on Middle Managers and Their Work , 1987, MIS Q..

[68]  Varun Grover,et al.  An Application of Expectancy Theory for Assessing User Motivation to Utilize an Expert System , 1992, J. Manag. Inf. Syst..

[69]  Adolph I. Katz.  MEASURING TECHNOLOGY’S BUSINESS VALUE: Organizations Seek to Prove IT Benefits , 1993 .

[70]  Albert H. Segars,et al.  Re-examining perceived ease of use and usefulness , 1993 .

[71]  S. K. Singh,et al.  Using information technology effectively: Organizational preparedness models , 1993, Inf. Manag..

[72]  Louis A. Le Blanc,et al.  An Empirical Investigation of the Relationship Between DSS Usage and System Performance: A Case Study of a Navigation Support System , 1990, MIS Q..

[73]  Kenneth L. Kraemer,et al.  The Usefulness of Computer-Based Information to Public Managers , 1993, MIS Q..

[74]  Wanda J. Orlikowski,et al.  A Short Form Measure of User Information Satisfaction: A Psychometric Evaluation and Notes on Use , 1987, J. Manag. Inf. Syst..

[75]  Brad Quinn Post,et al.  A Business Case Framework for Group Support Technology , 1992, J. Manag. Inf. Syst..

[76]  Charles H. Kriebel,et al.  A Method for Assessing the Economic Impact of Information Systems Technology on Organizations , 1985, ICIS.

[77]  Phillip Ein-Dor,et al.  Use of Management Information Systems: An Empirical Study , 1981, ICIS.

[78]  Blake Ives,et al.  The information system as a competitive weapon , 1984, CACM.

[79]  Lawrence W. Foster,et al.  Management Information Technology: Its Effects on Orgnizational Form and Function , 1984, MIS Q..

[80]  Steven W. Floyd,et al.  Path Analysis of the Relationship between Competitive Strategy, Information Technology, and Financial Performance , 1990, J. Manag. Inf. Syst..

[81]  Daniel Robey,et al.  User Attitudes and Management Information System Use , 1979 .

[82]  George P. Huber,et al.  A theory of the effects of advanced information technologies on organizational design, intelligence , 1990 .

[83]  Peter Weill,et al.  Managing Investment in Information Technology: Mini Case Examples and Implications , 1989, MIS Q..

[84]  Louis Raymond,et al.  Organizational Characteristics and MIS Success in the Context of Small Business , 1985, MIS Q..

[85]  Henri Barki,et al.  Rethinking the Concept of User Involvement , 1989, MIS Q..

[86]  M. Treacy,et al.  Utilization as a dependent variable in MIS research , 2015, DATB.

[87]  Izak Benbasat,et al.  An Empirical Investigation of Factors Influencing the Success of Customer-Oriented Strategic Systems , 1990, Inf. Syst. Res..

[88]  N. Melone.  A theoretical assessment of the user-satisfaction construct in information systems research , 1990 .

[89]  Eric D. Carlson,et al.  Database Administration - Introduction , 1983, Data Base.

[90]  Vijay Sethi,et al.  Information technology and organizational performance: A critical evaluation of Computerworld's index of information systems effectiveness , 1993, Inf. Manag..

[91]  James F. Courtney,et al.  A Field Study of Organizational Factors Influencing DSS Success , 1985, MIS Q..

[92]  Peter B. Seddon,et al.  A Partial Test and Development of the DeLone and McLean Model of IS Success , 1994, ICIS.

[93]  R. Golembiewski.  Macro organizational behavior , 1980 .

[94]  Jonathan Miller,et al.  Measuring and aligning information systems with the organization: A case study , 1993, Inf. Manag..

[95]  George P. Huber,et al.  DECISION SUPPORT SYSTEMS FOR ILL‐STRUCTURED PROBLEMS: AN EMPIRICAL STUDY , 1987 .

[96]  Lee L. Gremillion.  Organization Size and Information System Use: An Empirical Study , 1984, J. Manag. Inf. Syst..

[97]  Richard W. Scamell,et al.  The Effects of Information System User Expectations on Their Performance and Perceptions , 1993, MIS Q..

[98]  A. B. Crawford,et al.  SIM Competition Paper: Corporate Electronic Mail - A Communication-Intensive Application of Information Technology , 1982, MIS Q..

[99]  William R. King,et al.  Evaluating Management Informatoin Systems , 1978, MIS Q..

[100]  Ananth Srinivasan,et al.  Alternative Measures of Systems Effectiveness: Associations and Implications , 1985, MIS Q..

[101]  Judy E. Scott.  The measurement of information systems effectiveness: evaluating a measuring instrument , 1995, DATB.

[102]  Tor Guimaraes,et al.  The Relationship Between User Participation and User Satisfaction: An Investigation of Four Contingency Factors , 1994, MIS Q..

[103]  Scott Hamilton,et al.  Evaluating Information Systems Effectiveness - Part I: Comparing Evaluation Approaches , 1981, MIS Q..

[104]  R. Bostrom,et al.  The impact of group support systems on group conflict and conflict management: an empirical investigation , 1993, [1993] Proceedings of the Twenty-sixth Hawaii International Conference on System Sciences.

[105]  C. Schewe.  The Management Information System User: An Exploratory Behavioral Analysis , 1976 .

[106]  Gordon B. Davis,et al.  Management information systems : conceptual foundations, structure, and development , 1985 .

[107]  Mo Adam Mahmood,et al.  Measuring the Organizational Impact of Information Technology Investment: An Exploratory Study , 1993, J. Manag. Inf. Syst..

[108]  T. Lincoln,et al.  Do computer systems really pay-off? , 1986, Inf. Manag..

[109]  Timothy Paul Cronan,et al.  On the Repeated Test-Retest Reliability of the End-User Computing Satisfaction Instrument: A Comment* , 1994 .

[110]  Mo Adam Mahmood,et al.  Impact of design methods on decision support systems success: An empirical assessment , 1985, Inf. Manag..