A learning-based method for combining testing techniques

This work presents a method to combine testing techniques adaptively during the testing process. It intends to mitigate the sources of uncertainty of software testing processes, by learning from past experience and, at the same time, adapting the technique selection to the current testing session. The method is based on machine learning strategies. It uses offline strategies to take historical information into account about the techniques performance collected in past testing sessions; then, online strategies are used to adapt the selection of test cases to the data observed as the testing proceeds. Experimental results show that techniques performance can be accurately characterized from features of the past testing sessions, by means of machine learning algorithms, and that integrating this result into the online algorithm allows improving the fault detection effectiveness with respect to single testing techniques, as well as to their random combination.

[1]  Kai-Yuan Cai,et al.  Adaptive software testing with fixed-memory feedback , 2007, J. Syst. Softw..

[2]  Elaine J. Weyuker,et al.  Predicting the location and number of faults in large software systems , 2005, IEEE Transactions on Software Engineering.

[3]  Jeffrey J. P. Tsai,et al.  Machine Learning and Software Engineering , 2004, Software Quality Journal.

[4]  Lionel C. Briand,et al.  Using Machine Learning to Support Debugging with Tarantula , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[5]  Akbar Siami Namin,et al.  Bayesian reasoning for software testing , 2010, FoSER '10.

[6]  Harald C. Gall,et al.  Cross-project defect prediction: a large scale experiment on data vs. domain vs. process , 2009, ESEC/SIGSOFT FSE.

[7]  Bev Littlewood,et al.  Modeling the Effects of Combining Diverse Software Fault Detection Techniques , 2000, IEEE Trans. Software Eng..

[8]  Geoff Holmes,et al.  Benchmarking Attribute Selection Techniques for Discrete Class Data Mining , 2003, IEEE Trans. Knowl. Data Eng..

[9]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[10]  Akbar Siami Namin,et al.  Prioritizing Mutation Operators using Probabilistic Sampling International Symposium on Software Reliability Engineering (ISSRE 2010) , 2010 .

[11]  Maliha S. Nash,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 2001, Technometrics.

[12]  Thomas G. Dietterich,et al.  Structured machine learning: the next ten years , 2008, Machine Learning.

[13]  Tim Menzies,et al.  Data Mining Static Code Attributes to Learn Defect Predictors , 2007, IEEE Transactions on Software Engineering.

[14]  Banu Diri,et al.  A systematic review of software fault prediction studies , 2009, Expert Syst. Appl..

[15]  Lionel C. Briand Novel Applications of Machine Learning in Software Testing , 2008, 2008 The Eighth International Conference on Quality Software.

[16]  Paul A. Strooper,et al.  An Iterative Empirical Strategy for the Systematic Selection of a Combination of Verification and Validation Technologies , 2007, Fifth International Workshop on Software Quality (WoSQ'07: ICSE Workshops 2007).

[17]  ZhangHongyu,et al.  Comments on "Data Mining Static Code Attributes to Learn Defect Predictors" , 2007 .

[18]  Bev Littlewood,et al.  Modelling the Effects of Combining Diverse Software Fault Detection Techniques , 2008, Formal Methods and Testing.

[19]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[20]  Stefan Wagner Software Quality Economics for Combining Defect-Detection Techniques⋆ , 2005 .

[21]  Lionel C. Briand,et al.  Using Machine Learning to Refine Black-Box Test Specifications and Test Suites , 2008, 2008 The Eighth International Conference on Quality Software.

[22]  Dieter Fox,et al.  Adapting the Sample Size in Particle Filters Through KLD-Sampling , 2003, Int. J. Robotics Res..

[23]  Michèle Sebag,et al.  A Machine Learning Approach for Statistical Software Testing , 2007, IJCAI.

[24]  Sira Vegas,et al.  A Characterisation Schema for Software Testing Techniques , 2005, Empirical Software Engineering.

[25]  Ian Witten,et al.  Data Mining , 2000 .

[26]  Du Zhang,et al.  APPLYING MACHINE LEARNING ALGORITHMS IN SOFTWARE DEVELOPMENT , 2000 .