Using Complexity Measures to Evaluate Software Development Projects: A Nonparametric Approach

In this article, we use newly developed complexity metrics for software development projects that are more useful than traditional measures such as lines of code and functional points. Next, we present an approach to assessing the relative efficiency of software projects using these complexity measures as outputs. Due to the nature of the complexity measures, the constant returns to scale assumption often used in data envelopment analysis (DEA) is not appropriate. We relax this assumption and estimate the DEA model assuming variable returns to scale. This two-step approach provides project managers with a decision support tool to assess project productivity, categorize projects, and evaluate critical success/failure factors in software development projects.

[1]  Sjaak Brinkkemper,et al.  Complexity Metrics for Systems Development Methods and Techniques , 1996, Inf. Syst..

[2]  Norman E. Fenton,et al.  Software Metrics: A Rigorous Approach , 1991 .

[3]  Jose L. Salmeron,et al.  Using the DEA methodology to rank software technical efficiency , 2005, CACM.

[4]  Ingunn Myrtveit,et al.  Identifying High Performance ERP Projects , 2003, IEEE Trans. Software Eng..

[5]  John Maleyeff,et al.  Quantitative Models for Performance Evaluation and Benchmarking: DEA with Spreadsheets and DEA Excel Solver , 2005 .

[6]  A. Boonstra,et al.  Does risk management contribute to IT project success? A meta-analysis of empirical evidence , 2010 .

[7]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[8]  Janice Thomas,et al.  Preparing project managers to deal with complexity–advanced project management education , 2008, IEEE Engineering Management Review.

[9]  Tyson R. Browning,et al.  Managing complex product development projects with design structure matrices and domain mapping matrices , 2007 .

[10]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[11]  Qing Cao,et al.  A case study approach for developing a project performance evaluation system , 2011 .

[12]  Jarmo J. Ahonen,et al.  Software development project success and failure from the supplier's perspective: A systematic literature review , 2012 .

[13]  Gad Vitner,et al.  Using data envelope analysis to compare project efficiency in a multi-project environment , 2006 .

[14]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[15]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[16]  Keng Siau,et al.  Unified Modeling Language: A Complexity Analysis , 2001, J. Database Manag..

[17]  June M. Verner,et al.  Why did your project fail? , 2009, Commun. ACM.

[18]  Khaled El Emam,et al.  A Replicated Survey of IT Software Project Failures , 2008, IEEE Software.

[19]  Audris Mockus,et al.  Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System , 2009, IEEE Transactions on Software Engineering.

[20]  Adedeji Badiru,et al.  Fuzzy Present Value Analysis Model For Evaluating Information System Projects , 2007 .

[21]  Rajiv D. Banker,et al.  A model to evaluate variables impacting the productivity of software maintenance projects , 1991 .

[22]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[23]  Joseph C. Paradi,et al.  Commercial branch performance evaluation and results communication in a Canadian bank--a DEA application , 2004, Eur. J. Oper. Res..