The Empirical Reality of IT Project Cost Overruns: Discovering A Power-Law Distribution

ABSTRACT If managers assume a normal or near-normal distribution of Information Technology (IT) project cost overruns, as is common, and cost overruns can be shown to follow a power-law distribution, managers may be unwittingly exposing their organizations to extreme risk by severely underestimating the probability of large cost overruns. In this research, we collect and analyze a large sample comprised of 5,392 IT projects to empirically examine the probability distribution of IT project cost overruns. Further, we propose and examine a mechanism that can explain such a distribution. Our results reveal that IT projects are far riskier in terms of cost than normally assumed by decision makers and scholars. Specifically, we found that IT project cost overruns follow a power-law distribution in which there are a large number of projects with relatively small overruns and a fat tail that includes a smaller number of projects with extreme overruns. A possible generative mechanism for the identified power-law distribution is found in interdependencies among technological components in IT systems. We propose and demonstrate, through computer simulation, that a problem in a single technological component can lead to chain reactions in which other interdependent components are affected, causing substantial overruns. What the power law tells us is that extreme IT project cost overruns will occur and that the prevalence of these will be grossly underestimated if managers assume that overruns follow a normal or near-normal distribution. This underscores the importance of realistically assessing and mitigating the cost risk of new IT projects up front.

[1]  R. Armentano,et al.  Internet of Things and Artificial Intelligence in Healthcare During COVID-19 Pandemic—A South American Perspective , 2020, Frontiers in Public Health.

[2]  P. Scuffham,et al.  The Internet of Things: Impact and Implications for Health Care Delivery , 2020, Journal of medical Internet research.

[3]  B. Flyvbjerg The law of regression to the tail: How to survive Covid-19, the climate crisis, and other disasters , 2020, Environmental Science & Policy.

[4]  Mark Keil,et al.  Detection of early warning signals for overruns in IS projects: linguistic analysis of business case language , 2020, Eur. J. Inf. Syst..

[5]  Mark Keil,et al.  Seeing the Trees or the Forest? The Effect of IT Project Managers' Mental Construal on IT Project Risk Management Activities , 2019, Inf. Syst. Res..

[6]  Anupam Haldar,et al.  A framework for managing uncertainty in information system project selection: an intelligent fuzzy approach , 2019, International Journal of Management Science and Engineering Management.

[7]  Xiaojun Zhang,et al.  A Risk Mitigation Framework for Information Technology Projects: A Cultural Contingency Perspective , 2019, J. Manag. Inf. Syst..

[8]  Nilanjan Dey,et al.  Internet of Things and Big Data Analytics Toward Next-Generation Intelligence , 2018 .

[9]  Rudy Hirschheim,et al.  An Agile Methodology for the Disaster Recovery of Information Systems Under Catastrophic Scenarios , 2017, J. Manag. Inf. Syst..

[10]  Jerker Denrell Sampling Biases Explain Decision Biases , 2017 .

[11]  Stephan Aier,et al.  What Drives Application Portfolio Complexity? An Empirical Analysis of Application Portfolio Cost Drivers at a Global Automotive Company , 2016, 2016 IEEE 18th Conference on Business Informatics (CBI).

[12]  Carliss Y. Baldwin,et al.  The Mirroring Hypothesis: Theory, Evidence and Exceptions , 2016 .

[13]  Herman Aguinis,et al.  CUMULATIVE ADVANTAGE: CONDUCTORS AND INSULATORS OF HEAVY-TAILED PRODUCTIVITY DISTRIBUTIONS AND PRODUCTIVITY STARS , 2016 .

[14]  Christopher G. Reddick,et al.  Why e-government projects fail? An analysis of the Healthcare.gov website , 2016, Gov. Inf. Q..

[15]  In Lee,et al.  The Internet of Things (IoT): Applications, investments, and challenges for enterprises , 2015 .

[16]  Amrit Tiwana,et al.  Evolutionary Competition in Platform Ecosystems , 2015, Inf. Syst. Res..

[17]  Tridas Mukhopadhyay,et al.  Project Managers’ Practical Intelligence and Project Performance in Software Offshore Outsourcing: A Field Study , 2014, Inf. Syst. Res..

[18]  R. Marks Learning to be risk averse? , 2014, 2014 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr).

[19]  Peerasit Patanakul,et al.  Managing large-scale IS/IT projects in the public sector: Problems and causes leading to poor performance , 2014 .

[20]  Shan Liu,et al.  How user risk and requirements risk moderate the effects of formal and informal control on the process performance of IT projects , 2013, Eur. J. Inf. Syst..

[21]  Cheng Zhang,et al.  Risk Mitigation in Supply Chain Digitization: System Modularity and Information Technology Governance , 2013, J. Manag. Inf. Syst..

[22]  Magne Jørgensen,et al.  How does project size affect cost estimation error? Statistical artifacts and methodological challenges , 2012 .

[23]  D. Sornette,et al.  Dragon-kings: Mechanisms, statistical methods and empirical evidence , 2012, 1205.1002.

[24]  Param Vir Singh,et al.  Network Effects: The Influence of Structural Capital on Open Source Project Success , 2011, MIS Q..

[25]  Christopher L. Magee,et al.  Engineering Systems: Meeting Human Needs in a Complex Technological World , 2011 .

[26]  C. Perrow Normal Accidents: Living with High Risk Technologies - Updated Edition , 2011 .

[27]  Karl T. Ulrich,et al.  The role of product architecture in the manufacturing firm , 2011 .

[28]  B. Flyvbjerg,et al.  Why Your IT Project May Be Riskier than You Think , 2011, 1304.0265.

[29]  Bill McKelvey,et al.  Connectivity, Extremes, and Adaptation: A Power-Law Perspective of Organizational Effectiveness , 2011 .

[30]  Anandasivam Gopal,et al.  Research Note - The Role of Organizational Controls and Boundary Spanning in Software Development Outsourcing: Implications for Project Performance , 2010, Inf. Syst. Res..

[31]  Kieran Conboy,et al.  Project failure en masse: a study of loose budgetary control in ISD projects , 2010, Eur. J. Inf. Syst..

[32]  Bill McKelvey,et al.  From Gaussian to Paretian Thinking: Causes and Implications of Power Laws in Organizations , 2009 .

[33]  Likoebe M. Maruping,et al.  Offshore information systems project success: the role of social embeddedness and cultural characteristics , 2009 .

[34]  Amrit Tiwana,et al.  Governance-Knowledge Fit in Systems Development Projects , 2009, Inf. Syst. Res..

[35]  Martin Mocker,et al.  What Is Complex About 273 Applications? Untangling Application Architecture Complexity in a Case of European Investment Banking , 2009, 2009 42nd Hawaii International Conference on System Sciences.

[36]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[37]  Blaize Horner Reich,et al.  A Temporal Model of Information Technology Project Performance , 2007, J. Manag. Inf. Syst..

[38]  Robert J. Kauffman,et al.  Option-Based Risk Management: A Field Study of Sequential Information Technology Investment Decisions , 2007, J. Manag. Inf. Syst..

[39]  Nassim Nicholas Taleb,et al.  The Black Swan: The Impact of the Highly Improbable , 2007 .

[40]  Todd Little,et al.  Schedule estimation and uncertainty surrounding the cone of uncertainty , 2006, IEEE Software.

[41]  Gary Klein,et al.  The Effects of User Partnering and User Non-Support on Project Performance , 2006, J. Assoc. Inf. Syst..

[42]  Magne Jørgensen,et al.  A comparison of software project overruns - flexible versus sequential development models , 2005, IEEE Transactions on Software Engineering.

[43]  M. Gallegati,et al.  Pareto's Law of Income Distribution: Evidence for Germany, the United Kingdom, and the United States , 2005, physics/0504217.

[44]  Weidong Xia,et al.  Complexity of Information Systems Development Projects: Conceptualization and Measurement Development , 2005, J. Manag. Inf. Syst..

[45]  M. Newman Power laws, Pareto distributions and Zipf's law , 2005 .

[46]  Indranil R. Bardhan,et al.  Prioritizing a Portfolio of Information Technology Investment Projects , 2004, J. Manag. Inf. Syst..

[47]  R. Hertwig,et al.  Decisions from Experience and the Effect of Rare Events in Risky Choice , 2004, Psychological science.

[48]  Daniel A. Levinthal,et al.  Modularity and Innovation in Complex Systems , 2002, Manag. Sci..

[49]  Dean Keith Simonton,et al.  Scientific creativity as constrained stochastic behavior: the integration of product, person, and process perspectives. , 2003, Psychological bulletin.

[50]  Olivier L. de Weck ENGINEERING SYSTEMS , 2002 .

[51]  J. March,et al.  Adaptation as Information Restriction: The Hot Stove Effect , 2001 .

[52]  Suzanne Rivard,et al.  An Integrative Contingency Model of Software Project Risk Management , 2001, J. Manag. Inf. Syst..

[53]  Mark Keil,et al.  Why Software Projects Escalate: An Empirical Analysis and Test of Four Theoretical Models , 2000, MIS Q..

[54]  Kim B. Clark,et al.  Design Rules: The Power of Modularity , 2000 .

[55]  D. Turcotte,et al.  Self-organized criticality , 1999 .

[56]  Guido Boffetta,et al.  Power Laws in Solar Flares: Self-Organized Criticality or Turbulence? , 1999, chao-dyn/9904043.

[57]  Victoria L. Mitchell,et al.  The Effects of Coupling it and Work Process Strategies in Redesign Projects , 1999 .

[58]  Jan Venselaar,et al.  DESIGN RULES , 1999 .

[59]  D. Turcotte,et al.  Forest fires: An example of self-organized critical behavior , 1998, Science.

[60]  Henrik Jeldtoft Jensen,et al.  Self-Organized Criticality: Emergent Complex Behavior in Physical and Biological Systems , 1998 .

[61]  David M. Raup,et al.  How Nature Works: The Science of Self-Organized Criticality , 1997 .

[62]  Ram L. Kumar,et al.  A Note on Project Risk and Option Values of Investments in Information Technologies , 1996, J. Manag. Inf. Syst..

[63]  Per Bak,et al.  How Nature Works , 1996 .

[64]  Cynthia Mathis Beath,et al.  The enactments and consequences of token, shared, and compliant participation in information systems development , 1996 .

[65]  Sarma R. Nidumolu The Effect of Coordination and Uncertainty on Software Project Performance: Residual Performance Risk as an Intervening Variable , 1995, Inf. Syst. Res..

[66]  D. Kahneman,et al.  Timid choices and bold forecasts: a cognitive perspective on risk taking , 1993 .

[67]  François Bergeron,et al.  Estimation of information systems development efforts: A pilot study , 1992, Inf. Manag..

[68]  HERBERT A. SIMON,et al.  The Architecture of Complexity , 1991 .

[69]  Q. Vuong Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses , 1989 .

[70]  Justus D. Naumann,et al.  Empirical investigation of systems development practices and results , 1984, Inf. Manag..

[71]  James D. McKeen Successful Development Strategies for Business Application Systems , 1983, MIS Q..

[72]  Robert W. Zmud,et al.  Management of Large Software Development Efforts , 1980, MIS Q..

[73]  R. Ackoff Towards a System of Systems Concepts , 1971 .

[74]  E. Fama The Behavior of Stock-Market Prices , 1965 .

[75]  B. Mandelbrot New Methods in Statistical Economics , 1963, Journal of Political Economy.