Using Classification Trees to Predict Performance in Information Technology Projects

Classification trees are introduced as a modeling technique to predict IT project performance. A comparative analysis of classification tree, regression and neural network techniques provides promising evidence that classification trees can provide more actionable output for project decisions. The relative accuracy from classification trees, vis-à-vis regression and neural network techniques is demonstrated using a data set of 440 projects including 8 performance factors and a binary dependent variable of performance. Results suggest classification tree techniques provide comparable, and possibly superior, predictive capabilities. We conclude that a performance assessment based on classification trees could provide an effective decision tool for managing IT project performance.

[1]  Kyung Hoon Yang,et al.  Human decision-making behavior and modeling effects , 2008, Decis. Support Syst..

[2]  Kalle Lyytinen,et al.  Attention Shaping and Software Risk - A Categorical Analysis of Four Classical Risk Management Approaches , 1998, Inf. Syst. Res..

[3]  J Ropponen,et al.  Can software risk management improve system development: an exploratory study , 1997 .

[4]  Peter C. Verhoef,et al.  The commercial use of segmentation and predictive modeling techniques for database marketing in the Netherlands , 2003, Decis. Support Syst..

[5]  Padhraic Smyth,et al.  Business applications of data mining , 2002, CACM.

[6]  Denis F. Cioffi.  Completing projects according to plans: an earned-value improvement index , 2006, J. Oper. Res. Soc..

[7]  Mark Keil,et al.  Understanding software project risk: a cluster analysis , 2004, Inf. Manag..

[8]  Thomas Reinartz,et al.  CRISP-DM 1.0: Step-by-step data mining guide , 2000 .

[9]  Amrit Tiwana,et al.  The one-minute risk assessment tool , 2004, CACM.

[10]  Denis F. Cioffi,et al.  A practical method of determining project risk contingency budgets , 2009, J. Oper. Res. Soc..

[11]  K. S. Sarma,et al.  Predictive Modeling With SAS Enterprise Miner: Practical Solutions for Business Applications , 2007 .

[12]  Blaize Horner Reich,et al.  The impact of size and volatility on IT project performance , 2007, CACM.

[13]  Sarma R. Nidumolu.  A Comparison of the Structural Contingency and Risk-Based Perspectives on Coordination in Software Development Projects , 1996, J. Manag. Inf. Syst..

[14]  William J. Long,et al.  Using Classification Tree and Logistic Regression Methods to Diagnose Myocardial Infarction , 1998, MedInfo.

[15]  Kalle Lyytinen,et al.  A framework for identifying software project risks , 1998, CACM.

[16]  F. W. McFarlan,et al.  Portfolio approach to information systems , 1989 .

[17]  Patricia B. Cerrito.  Introduction to Data Mining Using SAS Enterprise Miner , 2006 .

[18]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[19]  Mark Keil,et al.  How Software Project Risk Affects Project Performance: An Investigation of the Dimensions of Risk and an Exploratory Model , 2004, Decis. Sci..

[20]  Kalle Lyytinen,et al.  Components of Software Development Risk: How to Address Them? A Project Manager Survey , 2000, IEEE Trans. Software Eng..

[21]  Martine R. Haas,et al.  Knowledge Gathering, Team Capabilities, and Project Performance in Challenging Work Environments , 2006, Manag. Sci..

[22]  Heikki Topi,et al.  A Review of Software Packages for Data Mining , 2003 .

[23]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[24]  María N. Moreno García,et al.  Building knowledge discovery-driven models for decision support in project management , 2004, Decis. Support Syst..

[25]  Mark Keil,et al.  Software project risks and their effect on outcomes , 2004, CACM.

[26]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[27]  Mark Keil,et al.  Predicting information technology project escalation: A neural network approach , 2003, Eur. J. Oper. Res..

[28]  Mario Vanhoucke,et al.  A simulation and evaluation of earned value metrics to forecast the project duration , 2005, J. Oper. Res. Soc..

[29]  R. Lewis.  An Introduction to Classification and Regression Tree (CART) Analysis , 2000 .

[30]  ReichBlaize Horner,et al.  The impact of size and volatility on IT project performance , 2007 .

[31]  Suzanne Rivard,et al.  An Integrative Contingency Model of Software Project Risk Management , 2001, J. Manag. Inf. Syst..

[32]  Sarma R. Nidumolu.  The Effect of Coordination and Uncertainty on Software Project Performance: Residual Performance Risk as an Intervening Variable , 1995, Inf. Syst. Res..