Practical considerations in deploying AI for defect prediction: a case study within the Turkish telecommunication industry

We have conducted a study in a large telecommunication company in Turkey to employ a software measurement program and to predict pre-release defects. We have previously built such predictors using AI techniques. This project is a transfer of our research experience into a real life setting to solve a specific problem for the company: to improve code quality by predicting pre-release defects and efficiently allocating testing resources. Our results in this project have many practical implications that managers have started benefiting: code analysis, bug tracking, effective use of version management system and defect prediction. Using version history information, developers can find around 88% of the defects with 28% false alarms, compared to same detection rate with 50% false alarms without using historical data. In this paper we also shared in detail our experience in terms of the project steps (i.e. challenges and opportunities).

[1]  Victor R. Basili,et al.  The influence of organizational structure on software quality , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[2]  Ayse Basar Bener,et al.  Analysis of Naive Bayes' assumptions on software fault data: An empirical study , 2009, Data Knowl. Eng..

[3]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[4]  Bart Baesens,et al.  Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings , 2008, IEEE Transactions on Software Engineering.

[5]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[6]  Günther Ruhe,et al.  Optimized Resource Allocation for Software Release Planning , 2009, IEEE Transactions on Software Engineering.

[7]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[8]  Brendan Murphy,et al.  Using Historical In-Process and Product Metrics for Early Estimation of Software Failures , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[9]  Vyacheslav P. Tuzlukov,et al.  Signal detection theory , 2001 .

[10]  Burak Turhan,et al.  Implications of ceiling effects in defect predictors , 2008, PROMISE '08.

[11]  Elaine J. Weyuker,et al.  Predicting the location and number of faults in large software systems , 2005, IEEE Transactions on Software Engineering.

[12]  Ayse Basar Bener,et al.  On the relative value of cross-company and within-company data for defect prediction , 2009, Empirical Software Engineering.

[13]  Tim Menzies,et al.  Data Mining Static Code Attributes to Learn Defect Predictors , 2007, IEEE Transactions on Software Engineering.

[14]  Ayse Basar Bener,et al.  Nearest neighbor sampling for cross company defect predictors: abstract only , 2008, DEFECTS '08.

[15]  Lionel C. Briand,et al.  Predicting fault-prone components in a java legacy system , 2006, ISESE '06.

[16]  William Marsh,et al.  On the effectiveness of early life cycle defect prediction with Bayesian Nets , 2008, Empirical Software Engineering.

[17]  Ayse Basar Bener,et al.  Ensemble of software defect predictors: a case study , 2008, ESEM '08.

[18]  Hongfang Liu,et al.  Building effective defect-prediction models in practice , 2005, IEEE Software.

[19]  Maurice H. Halstead,et al.  Elements of software science (Operating and programming systems series) , 1977 .

[20]  Thomas J. Ostrand,et al.  \{PROMISE\} Repository of empirical software engineering data , 2007 .