The Impact of Business Intelligence Tools on Performance: A User Satisfaction Paradox?

While Business Intelligence (BI) initiatives have been a top-priority of CIOs around the world for several years, accounting for billions of USD of IT investments per annum (IDC), academic research on the actual benefits derived from BI tools and the drivers of these benefits remain sparse. This paper reports the findings of an exploratory, cross-sectional field study investigating the factors that define and drive benefits associated with the deployment of dedicated BI tools. BI is broadly defined as an analytical process which transforms fragmented data of enterprises and markets into action-oriented information or knowledge about objectives, opportunities and positions of an organization; BI tools are software products primarily designed and deployed to support this analytical process (e.g. data warehouse software, data mining software, digital dashboards applications). Building upon DeLoneand McLean’s (1992; 2002; 2003) information systems success model, we develop, test and refine a BI quality and performance model adapted for the specific purpose, application, user group and technology of BI tools. The ultimate performance predictors in this model are user satisfaction and the impact of BI tools on managerial decision quality, both of which are determined by data quality. Partial Least Square (PLS) modeling is used to analyze data collected in a survey administered to IT executives of large Australian Stock Exchange (ASX) listed companies. The results confirm some of the theoretical relationships established in – especially the original – DeLone-McLean model in the specific context of BI. More importantly, the results also confirm the important role of explicit BI management as antecedent of benefits derived from BI tools, and the key impact of data quality on managerial decision making and organizational performance. However, the results also reveal a ‘user satisfaction paradox’: In contrast to the predictions derived from the DeLone-McLean model, organizational performance is negatively associated with user satisfaction with BI tools. Financial performance data collected for ex-post verification of this unexpected result confirm this paradox. We discuss BI-specific interpretations of these unexpected findings and provide avenues for future research.

[1]  Marvin Moser The More Things Change, the More They Remain the Same , 2009, Journal of clinical hypertension.

[2]  David C. Yen,et al.  Critical factors influencing the adoption of data warehouse technology: a study of the banking industry in Taiwan , 2004, Decis. Support Syst..

[3]  Eric M. Olson,et al.  Strategy type and performance : The influence of sales force management , 2000 .

[4]  Thomas D. Clark,et al.  The Dynamic Structure of Management Support Systems: Theory Development, Research Focus, and Direction , 2007, MIS Q..

[5]  Hugh J. Watson,et al.  Tutorial: Business Intelligence - Past, Present, and Future , 2009, Communications of the Association for Information Systems.

[6]  Petti Van Rekom Data warehouse : a case study of the factors effecting user satisfaction , 2002 .

[7]  Herman Wold,et al.  Soft modelling: The Basic Design and Some Extensions , 1982 .

[8]  Hans-Georg Kemper,et al.  Management Support with Structured and Unstructured Data—An Integrated Business Intelligence Framework , 2008, Inf. Syst. Manag..

[9]  Alan R. Hevner,et al.  Integrated decision support systems: A data warehousing perspective , 2007, Decis. Support Syst..

[10]  Peter Chamoni,et al.  Integrationstrends bei Business-Intelligence-Systemen , 2004, Wirtschaftsinf..

[11]  P. Shrout,et al.  Mediation in experimental and nonexperimental studies: new procedures and recommendations. , 2002, Psychological methods.

[12]  E. Ziegel,et al.  Balanced Scorecard , 2019, Encyclopedia of Public Administration and Public Policy, Third Edition.

[13]  Beth A. Latshaw The More Things Change, the More They Remain the Same? , 2011 .

[14]  Andrew Stein,et al.  Business intelligence solution evolution: adoption and use , 2005 .

[15]  Wynne W. Chin The partial least squares approach for structural equation modeling. , 1998 .

[16]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[17]  Barbara Wixom Business Intelligence Software for the Classroom: MicroStrategy Resources on the Teradata University Network , 2004, Commun. Assoc. Inf. Syst..

[18]  Judea Pearl,et al.  Direct and Indirect Effects , 2001, UAI.

[19]  Barbara Wixom,et al.  Continental Airlines Continues to Soar with Business Intelligence , 2008, Inf. Syst. Manag..

[20]  Graham Pervan,et al.  The Use of Business Intelligence Systems in Australia , 2008, ACIS.

[21]  J. Schumpeter,et al.  Capitalism, Socialism and Democracy , 1943 .

[22]  Frédéric Adam,et al.  Benefit realisation through ERP: the re-emergence of data warehousing , 2003 .

[23]  Bruce A. Walters,et al.  Chief executive scanning emphases, environmental dynamism, and manufacturing firm performance , 2003 .

[24]  G. Hertel,et al.  Do Shy People Prefer to Send E-Mail? Personality Effects on Communication Media Preferences in Threatening and Nonthreatening Situations , 2008 .

[25]  Criston W. Cox,et al.  Balancing decision speed and decision quality: Assessing the impact of Business Intelligence Systems in high velocity environments , 2010 .

[26]  Barbara Wixom,et al.  The benefits of data warehousing: why some organizations realize exceptional payoffs , 2002, Inf. Manag..

[27]  Barbara Wixom,et al.  Current Practices in Data Warehousing , 2001, Inf. Syst. Manag..

[28]  David C. Yen,et al.  Enterprise integration with advanced information technologies: ERP and data warehousing , 2003, Inf. Manag. Comput. Secur..

[29]  M. Lessnoff Capitalism, Socialism and Democracy , 1979 .

[30]  Adir Even,et al.  Managing Metadata in Data Warehouses: Pitfalls and Possibilities , 2004, Commun. Assoc. Inf. Syst..

[31]  Markus Grünwald,et al.  Business Intelligence , 2009, Informatik-Spektrum.

[32]  Ephraim R. McLean,et al.  Information systems success revisited , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[33]  V. Govindarajan,et al.  Strategy, Control Systems, and Resource Sharing: Effects on Business-Unit Performance , 1990 .

[34]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[35]  Richard J. Arend Revisiting the logical and research considerations of competitive advantage , 2003 .

[36]  George S. Yip,et al.  Exploiting globalization potential: U.S. and japanese strategies , 1994 .

[37]  Barbara Wixom,et al.  Antecedents of Information and System Quality: An Empirical Examination Within the Context of Data Warehousing , 2005, J. Manag. Inf. Syst..

[38]  Frédéric Adam,et al.  Towards a model of organisational prerequisites for enterprise-wide systems integration: Examining ERP and data warehousing , 2005, J. Enterp. Inf. Manag..

[39]  Thomas H. Davenport,et al.  Business Intelligence and Organizational Decisions , 2010, Int. J. Bus. Intell. Res..

[40]  Stephen R. Gardner Building the data warehouse , 1998, CACM.

[41]  K. Bollen,et al.  DIRECT AND INDIRECT EFFECTS: CLASSICAL AND BOOTSTRAP ESTIMATES OF VARIABILITY , 1990 .

[42]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[43]  R. Sohi,et al.  IT competency and firm performance: is organizational learning a missing link? , 2003 .

[44]  Peter Drucker,et al.  A Brief History of Decision Support Systems , 2006 .

[45]  Michael J. Davern,et al.  Measuring the effects of business intelligence systems: The relationship between business process and organizational performance , 2008, Int. J. Account. Inf. Syst..

[46]  George S. Yip,et al.  EXPLOITING GLOBALIZATION POTENTIAL: U.S. , 1994 .

[47]  Mark I. Hwang,et al.  A Structural Model of Data Warehousing Success , 2008, J. Comput. Inf. Syst..

[48]  P. Shrout,et al.  Mediation in experimental and nonexperimental studies: new procedures and recommendations. , 2002, Psychological methods.

[49]  Jurij Jaklič,et al.  Assessing Benefits of Business Intelligence Systems – A Case Study , 2010 .

[50]  Benita M Gullkvist Drivers of Change in Management Accounting Practices in an ERP Environment , 2012 .

[51]  Barbara Wixom,et al.  An Empirical Investigation of the Factors Affecting Data Warehousing Success , 2001, MIS Q..

[52]  Frank S. C. Tseng,et al.  The concept of document warehousing for multi-dimensional modeling of textual-based business intelligence , 2006, Decis. Support Syst..

[53]  Martin D. Solomon Ensuring A Successful Data Warehouse Initiative , 2005, Inf. Syst. Manag..

[54]  M. Oss What do customers really want? , 2006, Behavioral healthcare.

[55]  John Hulland,et al.  Use of partial least squares (PLS) in strategic management research: a review of four recent studies , 1999 .

[56]  E. Ziegel,et al.  The Balanced Scorecard , 1998 .

[57]  Herman Wold,et al.  Systems under indirect observation : causality, structure, prediction , 1982 .

[58]  R. Rumelt Theory, Strategy, and Entrepreneurship , 2005 .

[59]  Hans Peter Luhn,et al.  A Business Intelligence System , 1958, IBM J. Res. Dev..

[60]  Barbara Wixom,et al.  The BI-Based Organization , 2010, Int. J. Bus. Intell. Res..

[61]  David James Power,et al.  A brief history of decision support systems , 2003, WWW 2003.

[62]  Lei-da Chen,et al.  Measuring user satisfaction with data warehouses: an exploratory study , 2000, Inf. Manag..

[63]  Ephraim R. McLean,et al.  A meta-analytic assessment of the DeLone and McLean IS success model: An examination of IS success at the individual level , 2009, Inf. Manag..

[64]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[65]  Yong-Tae Park,et al.  An empirical investigation of the effects of data warehousing on decision performance , 2006, Inf. Manag..

[66]  Gus Welty WHAT DO CUSTOMERS REALLY WANT , 1994 .

[67]  Wynne W. Chin,et al.  An Introduction to a Permutation Based Procedure for Multi-Group PLS Analysis: Results of Tests of Differences on Simulated Data and a Cross Cultural Analysis of the Sourcing of Information System Services Between Germany and the USA , 2010 .

[68]  Antti Lönnqvist,et al.  The Measurement of Business Intelligence , 2005, Inf. Syst. Manag..