Measuring organizational IS effectiveness: an overview and update of senior management perspectives

This paper reports the views of 80 senior IT managers about IT evaluation approaches, and the benefits that IT provides for their organizations. Their views were obtained through a survey mailed to medium to large organizations in both Europe and the US. The survey sought answers to three questions: How does the senior IT manager's organization assess the value of its (1) overall IT investment portfolio? (2) individual IT projects and applications? (3) IT function? Questions for the survey were based on a thorough review of prior research into IT evaluation. The paper provides a summary of that prior work and compares current findings to prior research. Consistent with prior research, the paper concludes that many firms do not conduct rigorous evaluations of all their IT investments, and offers a tentative explanation of why it might be rational to skip formal evaluations in some circumstances. It also posits five push factors to explain occasions when more lasting improvements in IT evaluation practice have occurred.

[1]  D. Remenyi,et al.  Effective Measurement and Management of It Costs and Benefits: Key Issues , 1995 .

[2]  Peter G.W. Keen,et al.  Shaping the Future: Business Design Through Information Technology , 1991 .

[3]  Stephanie Lester,et al.  Information Economics: linking business performance to information technology , 1990, J. Inf. Technol..

[4]  William J. Kettinger,et al.  Perceived Service Quality and User Satisfaction with the Information Services Function , 1994 .

[5]  C. James Bacon,et al.  The Use of Decision Criteria in Selecting Information Systems/Technology Investments , 1992, MIS Q..

[6]  N. Venkatraman.  Beyond Outsourcing: Managing IT Resources as a Value Center , 1997 .

[7]  Sunder Kekre,et al.  Business Value of Information Technology: A Study of Electronic Data Interchange , 1995, MIS Q..

[8]  Carol V. Brown.  Horizontal Mechanisms Under Differing IS Organization Contexts , 1999, MIS Q..

[9]  L. Willcocks,et al.  Managing I. T. as a Strategic Resource , 1997 .

[10]  Richard T. Watson,et al.  Measuring Information Systems Service Quality: Concerns for a Complete Canvas , 1997, MIS Q..

[11]  Frank Land,et al.  How to Assess Your It Investment: A Study of Methods and Practice , 1993 .

[12]  Leslie P. Willcocks,et al.  An Empirical Investigation of Information Technology Sourcing Practices: Lessons From Experience , 1998, MIS Q..

[13]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[14]  L. Willcocks,et al.  Strategic Sourcing of Information Systems: Perspectives and Practices , 1998 .

[15]  Michael J. Earl,et al.  Management Strategies for Information Technology , 1989 .

[16]  A. Parasuraman,et al.  SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. , 1988 .

[17]  Suprateek Sarker,et al.  IT-enabled organizational transformation: a case study of BPR failure at TELECO , 1999, J. Strateg. Inf. Syst..

[18]  Steve Smithson,et al.  Information systems evaluation in practice: a case study of organizational change , 2000, J. Inf. Technol..

[19]  P. Weill,et al.  Leveraging the New Infrastructure: How Market Leaders Capitalize on Information Technology , 1998 .

[20]  Leslie P. Willcocks,et al.  Global Information Technology Outsourcing: In Search of Business Advantage , 2001 .

[21]  Paul A. Strassmann,et al.  The Business Value of Computers: An Executive's Guide , 1990 .

[22]  Richard T. Watson,et al.  Measuring Information Systems Service Quality: Lessons From Two Longitudinal Case Studies , 1998, MIS Q..

[23]  Frank F. Land,et al.  Moving IS evaluation forward: learning themes and research issues , 1999, J. Strateg. Inf. Syst..

[24]  P. Strassmann.  The Squandered Computer: Evaluating the Business Alignment of Information Technologies , 1997 .

[25]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[26]  Margaret A. White,et al.  Shaping the future: Business design through information technology , 1992 .

[27]  Michael Sherwood-Smith,et al.  Achieving Maximum Value from Information Systems: A Process Approach , 1997 .

[28]  Mark Keil,et al.  Pulling the Plug: Software Project Management and the Problem of Project Escalation , 1995, MIS Q..

[29]  Mark Keil,et al.  Why Software Projects Escalate: An Empirical Analysis and Test of Four Theoretical Models , 2000, MIS Q..

[30]  Tom Renkema.  Boekbespreking Farbey, B.; Land, F.; Targett, D.: how to assess your IT investment, a study of methods and practice (Butterworth Heinemann, Oxford, 1993) , 1994 .

[31]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[32]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[33]  D. Sandy Staples,et al.  Dimensions of Information Systems Success , 1999, Commun. Assoc. Inf. Syst..

[34]  M. Parker,et al.  Information Economics: Linking Business Performance to Information Technology , 1988 .

[35]  M. Lynne Markus,et al.  Learning from adopters' experiences with ERP: problems encountered and success achieved , 2000, J. Inf. Technol..

[36]  Richard T. Watson,et al.  Service Quality: A Measure of Information System Effectiveness , 1995, MIS Q..

[37]  Peter Weill,et al.  Managing Investment in Information Technology: Mini Case Examples and Implications , 1989, MIS Q..

[38]  R. Klepper,et al.  Outsourcing information technology, systems and services , 1998 .

[39]  R. Hirschheim,et al.  Analysing information systems evaluation: another look at an old problem , 1998 .

[40]  F FarhoomandAli.  Review of "Corporate Information Systems Management: Text and Cases, Third Edition, by James I. Cash, Jr., F. W. McFarlan, James L. McKenney, and Lynda M. Applegate." Richard D. Irwin, Inc., 1992 , 1993 .

[41]  P. Strassmann.  The business value of computers , 1990 .

[42]  Leslie P. Willcocks.  Information management - the evaluation of information systems investments , 1994 .

[43]  M. Hammer,et al.  REENGINEERING THE CORPORATION: A MANIFESTO FOR BUSINESS REVOLUTION , 1995 .

[44]  M. Venkatraman.  It-enabled business transformation: from automation to business scope redefinition , 1994 .

[45]  R. Kaplan,et al.  The Balanced Scorecard: Translating Strategy into Action , 1996 .

[46]  William J. Kettinger,et al.  Pragmatic Perspectives on the Measurement of Information Systems Service Quality Analysis with LISREL : An Appendix to Pragmatic Perspectives on the Measurement of Information Systems Service Quality , 2002 .

[47]  A. Parasuraman,et al.  Reassessment of Expectations as a Comparison Standard in Measuring Service Quality: Implications for Further Research , 1994 .

[48]  Frank F. Land,et al.  Evaluating investments in IT , 1992, J. Inf. Technol..

[49]  John M. Ward,et al.  'Mind the Gap': diagnosing the relationship between the IT organisation and the rest of the business , 1999, J. Strateg. Inf. Syst..

[50]  Peter Weill,et al.  The Relationship Between Investment in Information Technology and Firm Performance: A Study of the Valve Manufacturing Sector , 1992, Inf. Syst. Res..

[51]  M. Morton,et al.  The corporation of the 1990s: Information technology and organizational transformation , 1993 .

[52]  Stephen King,et al.  Re-engineering in the face of a merger: soft systems and concurrent dynamics , 2000, J. Inf. Technol..

[53]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[54]  F. Caeldries.  Reengineering the Corporation: A Manifesto for Business Revolution , 1994 .

[55]  Anne Nagel,et al.  The Balanced Scorecard Translating Strategy Into Action , 2002 .

[56]  Albert H. Segars,et al.  Information systems effectiveness: The construct space and patters of application , 1996, Inf. Manag..

[57]  Joan A. Ballantine,et al.  Information systems/technology evaluation practices: evidence from UK organizations , 1996, J. Inf. Technol..

[58]  F. Warren McFarlan,et al.  Corporate information systems management : text and cases , 1988 .

[59]  M. Markus,et al.  The Enterprise System Experience— From Adoption to Success , 2000 .