The Development and Application of a Process-oriented "Thermometer" of IT Business Value

The issue of whether firms are receiving an adequate return on their investment in information technology (IT) continues to pervade managerial decision making. While productivity and other financial metrics are established hallmarks of IT investment evaluation, research has called for broader and richer metrics that can take into account the diversity of IT impacts. In this paper, we extend previous instrument development research to develop and test a process-oriented thermometer of IT business value using survey data based on executives’ perceptions of IT impacts at multiple points along the value chain. Consistent with earlier research, we find that our process measures are sensitive to differences in industry, firm size, and business strategy. Through additional analysis of post-implementation reviews of IT impacts in four firms, we find consistency of within-firm perceptual measures among teams of senior executives, highlighting the potential for our thermometer to gauge the level of IT impacts within a single firm. We conclude that process-oriented perceptual measures can offer new and useful insights into IT impacts, complementing what we already know from firm-level objective metrics.

[1]  R. T. Watson,et al.  Influences on the IS manager's perceptions of key issues: information scanning and the relationship with the CEO , 1990 .

[2]  Laura B. Cardinal,et al.  RETROSPECTIVE REPORTS IN ORGANIZATIONAL RESEARCH: A REEXAMINATION OF RECENT EVIDENCE , 1997 .

[3]  Rajiv Kohli,et al.  In Search of the Locus of Information Technology Business Value: Do measurement levels make a difference? , 2004 .

[4]  Charles F. Mason,et al.  How Supplier Characteristics Influence the Value of a Supplier Management ERP Application , 2002, Inf. Technol. Manag..

[5]  Gregory G. Dess,et al.  Measuring organizational performance in the absence of objective measures: The case of the privately-held firm and conglomerate business unit , 1984 .

[6]  Lorin M. Hitt,et al.  Beyond the productivity paradox , 1998, CACM.

[7]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[8]  Sabyasachi Mitra,et al.  Analyzing Cost-Effectiveness of Organizations: The Impact of Information Technology Spending , 1996, J. Manag. Inf. Syst..

[9]  Varun Grover,et al.  Strategic Information Systems Revisited: A Study in Sustainability and Performance , 1994, MIS Q..

[10]  Bryan Hattingh,et al.  The competitive advantage , 2007 .

[11]  Frederik D. Wiersema,et al.  The Discipline of Market Leaders , 1995 .

[12]  Arun Rai,et al.  Refocusing where and how IT value is realized: An empirical investigation , 1996 .

[13]  S. Harris,et al.  Organizational Performance and Information Technology Investment Intensity in the Insurance Industry , 1991 .

[14]  K. Weick.  FROM SENSEMAKING IN ORGANIZATIONS , 2021, The New Economic Sociology.

[15]  Stephan Kudyba,et al.  Research Report: Increasing Returns to Information Technology , 2002, Inf. Syst. Res..

[16]  Peter Weill,et al.  The Implications of Information Technology Infrastructure for Business Process Redesign , 1999, MIS Q..

[17]  Vijay Gurbaxani,et al.  A process oriented framework for assessing the business value of information technology , 1995, DATB.

[18]  Kevin Crowston,et al.  Assessing the Impact of Information Technology on Enterprise Level Performance , 1986, ICIS.

[19]  N. Carr.  IT doesn't matter , 2003, IEEE Engineering Management Review.

[20]  Cynthia Mathis Beath,et al.  Competitive Information Systems in Support of Pricing , 1986, ICIS.

[21]  Diane Wilson.  Assessing the impact of information technology on organizational performance , 1993 .

[22]  Kenneth L. Kraemer,et al.  Executives’ Perceptions of the Business Value of Information Technology: A Process-Oriented Approach , 2000, J. Manag. Inf. Syst..

[23]  R. Mason,et al.  Can information technology revitalize your customer service , 1990 .

[24]  E. Brynjolfsson,et al.  Beyond Computation: Information Technology, Organizational Transformation and Business Performance , 2000 .

[25]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[26]  Martin G. Curley,et al.  Managing Information Technology for Business Value: Practical Strategies for IT and Business Managers (IT Best Practices series) , 2004 .

[27]  Anandhi S. Bharadwaj,et al.  A Resource-Based Perspective on Information Technology Capability and Firm Performance: An Empirical Investigation , 2000, MIS Q..

[28]  Vijay Sethi,et al.  Development of measures to assess the extent to which an information technology application provides competitive advantage , 1994 .

[29]  Sundar G. Bharadwaj,et al.  Information Technology Effects on Firm Performance as Measured by Tobin's q , 1999 .

[30]  Yolande E. Chan.  IT Value: The Great Divide Between Qualitative and Quantitative and Individual and Organizational Measures , 2000, J. Manag. Inf. Syst..

[31]  Robert J. Kauffman,et al.  Modeling and Measuring the Business Value of Information Technology , 1988 .

[32]  Charles H. Kriebel,et al.  Measuring and Modeling the Business Value of Information Technology, Chapter 4. , 1988 .

[33]  Kenneth L. Kraemer,et al.  The Production of Information Services: A Firm-Level Analysis of Information Systems Budgets , 2000, Inf. Syst. Res..

[34]  P. M. Podsakoff,et al.  Self-Reports in Organizational Research: Problems and Prospects , 1986 .

[35]  Rajiv Kohli,et al.  Measuring Information Technology Payoff: A Meta - Analysis of Structural Variables in Firm - Level Empirical Research , 2003, Inf. Syst. Res..

[36]  Thomas H. Davenport,et al.  Process Innovation: Reengineering Work Through Information Technology , 1992 .

[37]  N. Venkatraman,et al.  Measurement of Business Economic Performance: An Examination of Method Convergence , 1987 .

[38]  Lorin M. Hitt,et al.  Productivity, Business Profitability, and Consumer Surplus: Three Different Measures of Information Technology Value , 1996, MIS Q..

[39]  Bijoy Bordoloi,et al.  A Framework for Assessing the Relationship between Information Technology Investments and Firm Performance , 2000, J. Manag. Inf. Syst..

[40]  Vernon J. Richardson,et al.  Returns on Investments in Information Technology: A Research Synthesis , 2002, J. Inf. Syst..

[41]  Rajiv Kohli,et al.  Information Technology Payoff in the Health-Care Industry: A Longitudinal Study , 2000, J. Manag. Inf. Syst..

[42]  Blake Ives,et al.  Executive Involvement and Participation in the Management of Information Technology , 1991, MIS Q..

[43]  John M. Mezias,et al.  Opening Pandora's Box: Studying the Accuracy of Managers' Perceptions , 1996 .

[44]  Kenneth L. Kraemer,et al.  Refining and Extending the Business Model with Information Technology: Dell Computer Corporation , 1999, Inf. Soc..

[45]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[46]  Gerard P. Hodgkinson,et al.  Re‐appraising Managers' Perceptual Errors: A Behavioural Decision‐Making Perspective , 2003 .

[47]  Maryellen R. Kelley,et al.  Productivity and Information Technology: The Elusive Connection , 1994 .

[48]  Kenneth L. Kraemer,et al.  Fact or Fiction? A Sensemaking Perspective on the Reality Behind Executives' Perceptions of IT Business Value , 2007, J. Manag. Inf. Syst..

[49]  Zahir Irani,et al.  The Propagation of Technology Management Taxonomies for Evaluating Investments in Information Systems , 2000, J. Manag. Inf. Syst..

[50]  Yannis Bakos,et al.  A Strategic Analysis of Electronic Marketplaces , 1991, MIS Q..

[51]  FRANK BANNISTER,et al.  Acts of faith: instinct, value and IT investment decisions , 2000, J. Inf. Technol..

[52]  Thomas W. Malone,et al.  MODELING COORDINATION IN ORGANIZATIONS AND MARKETS**Accepted by Richard M. Burton; received August 27, 1986. This paper has been with the author 2 months for 1 revision. , 1988 .

[53]  K. Srinivasan,et al.  Impact of Electronic Data Interchange Technology on JIT Shipments , 1994 .

[54]  J. Barney,et al.  CAPABILITIES, BUSINESS PROCESSES, AND COMPETITIVE ADVANTAGE: CHOOSING THE DEPENDENT VARIABLE IN EMPIRICAL TESTS OF THE RESOURCE-BASED VIEW , 2004 .

[55]  T. E. Marshall,et al.  Relating information technology investment to organizational performance: a causal model analysis , 1997 .

[56]  M. Porter,et al.  How Information Gives You Competitive Advantage , 1985 .

[57]  E. Brynjolfsson,et al.  Paradox Lost? Firm-Level Evidence on the Returns to Information Systems Spending , 1996 .

[58]  Øystein D. Fjeldstad,et al.  CONFIGURING VALUE FOR COMPETITIVE ADVANTAGE: ON CHAINS, SHOPS, AND NETWORKS , 1998 .

[59]  L. Richard Ye,et al.  Information technology and firm performance: Linking with environmental, strategic and managerial contexts , 1999, Inf. Manag..

[60]  Mark J. Safferstone.  Leveraging the New Infrastructure: How Market Leaders Capitalize on Information Technology , 1998 .

[61]  John M. Mezias,et al.  Studying the Accuracy of Managers' Perceptions: A Research Odyssey , 2003 .

[62]  Mo Adam Mahmood,et al.  A Comprehensive Model for Measuring the Potential Impact of Information Technology on Organizational Strategic Variables , 1991 .

[63]  Kenneth L. Kraemer,et al.  Review: Information Technology and Organizational Performance: An Integrative Model of IT Business Value , 2004, MIS Q..

[64]  Vijay Gurbaxani,et al.  The impact of information systems on organizations and markets , 1991, CACM.

[65]  Peter Weill,et al.  alignment: Learning , 2022 .

[66]  Kannan Srinivasan,et al.  Information Technology Impact on Process Output and Quality , 1997 .

[67]  Jooh Lee,et al.  Operational linkage between diverse dimensions of information technology investments and multifaceted aspects of a firm's economic performance , 2002, J. Inf. Technol..

[68]  Sidney E. Harris,et al.  Firm Size and the Information Technology Investment Intensity of LIfe Insurers , 1991, MIS Q..

[69]  Arik Ragowsky,et al.  Relating Benefits from Using IS to an Organization’s Operating Characteristics: Interpreting Results from Two Countries , 2000, J. Manag. Inf. Syst..

[70]  Kar Yan Tam,et al.  The Impact of Information Technology Investments on Firm Performance and Evaluation: Evidence from Newly Industrialized Economies , 1998, Inf. Syst. Res..

[71]  E. Brunswik.  Representative design and probabilistic theory in a functional psychology. , 1955, Psychological review.

[72]  Arun Rai,et al.  Technology investment and business performance , 1997, CACM.

[73]  Ken Peffers,et al.  The Impact of Information Technology Investment Announcements on the Market Value of the Firm , 1993, Inf. Syst. Res..

[74]  J. B. Quinn,et al.  Information technology: Increasing productivity in services , 1994 .

[75]  E. Brynjolfsson,et al.  Information Technology As A Factor Of Production: The Role Of Differences Among Firms , 1995 .

[76]  Sirkka L. Jarvenpaa,et al.  International Conference on Information Systems ( ICIS ) 1991 EXTERNAL VERSUS INTERNAL PERSPECTIVES IN DETERMINING A FIRM ' S PROGRESSIVE USE OF INFORMATION TECHNOLOGY , 2017 .

[77]  Robert J. Kauffman,et al.  An evaluative framework for research on the performance effects of information technology investment , 1989, ICIS '89.

[78]  Robert J. Kauffman,et al.  Measuring Gains in Operational Efficiency from Information Technology: A Study of the Positran Deployment at Hardee'S Inc. , 1990, J. Manag. Inf. Syst..

[79]  Chung-ki Min,et al.  The substitution of information technology for other factors of production: a Firm Level Analysis , 1997 .

[80]  Thiagarajan Ravichandran,et al.  Effect of Information Systems Resources and Capabilities on Firm Performance: A Resource-Based Perspective , 2005, J. Manag. Inf. Syst..

[81]  Louis Raymond,et al.  The Contribution of IT to the Bottom Line: A Contingency Perspective of Strategic Dimensions , 1995, ICIS.

[82]  Frances J. Milliken,et al.  Executives’ Perceptual Filters: What They Notice and How They Make Sense , 1988 .

[83]  Yannis Bakos,et al.  Information Technology and Corporate Strategy: A Research Perspective , 2015, MIS Q..

[84]  Leslie P. Willcocks,et al.  Measuring organizational IS effectiveness: an overview and update of senior management perspectives , 2002, DATB.

[85]  Namchul Shin,et al.  The impact of information technology on coordination costs: implications for firm productivity , 1997, ICIS '97.

[86]  Leslie P. Willcocks,et al.  Shaping the Future. Business Design Through Information Technology , 1991, J. Inf. Technol..

[87]  Blake Ives,et al.  The information system as a competitive weapon , 1984, CACM.

[88]  Steven W. Floyd,et al.  Path Analysis of the Relationship between Competitive Strategy, Information Technology, and Financial Performance , 1990, J. Manag. Inf. Syst..