Test Oracle Prediction for Mutation Based Fault Localization

In the process of software debugging, it is very critical and difficult to identify the locations of faults in an effective and accurate manner. Mutation-based fault localization (MBFL) is one of the most effective automated fault localization techniques that have been recently proposed, and it requires the execution results (passed or failed) of test cases to locate faults. One problem preventing MBFL from becoming a practical testing technique is the large amount of human effort involved, i.e., the test oracle problem, which refers to the process of checking an original program’s output of each test case. To mitigate the impact of this problem, we use mutant coverage information and learning algorithms to predict the oracle of the test cases in this paper. Empirical results show that the proposed method can reduce 80% of the human cost required to check the test oracles and achieve almost the same fault localization accuracy as compared to the original MBFL.

[1]  Jung-Hsien Chiang,et al.  Hierarchically SVM classification based on support vector clustering method and its application to document categorization , 2007, Expert Syst. Appl..

[2]  Yaxin Bi,et al.  KNN Model-Based Approach in Classification , 2003, OTM.

[3]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[4]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[5]  Arie van Deursen,et al.  A test-suite diagnosability metric for spectrum-based fault localization approaches , 2017, ICSE.

[6]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[7]  Baowen Xu,et al.  Code Coverage-Based Failure Proximity without Test Oracles , 2016, 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC).

[8]  Frank Tip,et al.  A survey of program slicing techniques , 1994, J. Program. Lang..

[9]  Yves Le Traon,et al.  Using Mutants to Locate "Unknown" Faults , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[10]  Kai-Yuan Cai,et al.  Exploring the usefulness of unlabelled test cases in software fault localization , 2018, J. Syst. Softw..

[11]  Zheng Li,et al.  Faster mutation-based fault localization with a novel mutation execution strategy , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[12]  Chen Xiang,et al.  Survey of Test Case Prioritization Techniques for Regression Testing , 2013 .

[13]  Mark Harman,et al.  The Oracle Problem in Software Testing: A Survey , 2015, IEEE Transactions on Software Engineering.

[14]  Yong Liu,et al.  An optimal mutation execution strategy for cost reduction of mutation-based fault localization , 2018, Inf. Sci..

[15]  Hadi Hemmati,et al.  Studying Test Case Failure Prediction for Test Case Prioritization , 2017, PROMISE.

[16]  Dominic Palmer-Brown,et al.  Hypercube Neural Network Algorithm for Classification , 2011, EANN/AIAI.

[17]  Yves Le Traon,et al.  Metallaxis‐FL: mutation‐based fault localization , 2015, Softw. Test. Verification Reliab..

[18]  Peter Zoeteweij,et al.  An Evaluation of Similarity Coefficients for Software Fault Localization , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[19]  Mohammad Shorif Uddin,et al.  Automated textile defect classification by Bayesian classifier based on statistical features , 2016, 2016 International Workshop on Computational Intelligence (IWCI).

[20]  Joseph Robert Horgan,et al.  Fault localization using execution slices and dataflow tests , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[21]  Giorgio Di Natale,et al.  Computing reliability: On the differences between software testing and software fault injection techniques , 2017, Microprocess. Microsystems.

[22]  Alessandro Oliveira Arantes,et al.  On Proposing a Test Oracle Generator Based on Static and Dynamic Source Code Analysis , 2015, QRS Companion.

[23]  Gwynne Lyons,et al.  Viewpoint: Policy Requirements for Protecting Wildlife from Endocrine Disruptors , 2005, Environmental health perspectives.

[24]  Yves Le Traon,et al.  Effective fault localization via mutation analysis: a selective mutation approach , 2014, SAC.

[25]  Ji-Hong Chen,et al.  Survey of Test Case Prioritization Techniques for Regression Testing: Survey of Test Case Prioritization Techniques for Regression Testing , 2014 .

[26]  G. Ruxton.  The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test , 2006 .

[27]  Seiichi Nakagawa,et al.  A deep neural network integrated with filterbank learning for speech recognition , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[29]  José Carlos Maldonado,et al.  Proteum/IM 2.0: An Integrated Mutation Testing Environment , 2001 .

[30]  David Leon,et al.  A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[31]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[32]  Andreas Zeller,et al.  Lightweight bug localization with AMPLE , 2005, AADEBUG'05.

[33]  James M. Conrad,et al.  The effects of pre-filtering and individualizing components for electroencephalography neural network classification , 2017, SoutheastCon 2017.

[34]  Frank Storbeck,et al.  Fish species recognition using computer vision and a neural network , 2001 .

[35]  William E. Howden,et al.  Theoretical and Empirical Studies of Program Testing , 1978, IEEE Transactions on Software Engineering.

[36]  B. Efron.  Student's t-Test under Symmetry Conditions , 1969 .

[37]  Yong Liu,et al.  Statement-Oriented Mutant Reduction Strategy for Mutation Based Fault Localization , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[38]  Wes Masri,et al.  Fault localization based on information flow coverage , 2010, Softw. Test. Verification Reliab..

[39]  Yong Liu,et al.  Fault Classification Oriented Spectrum Based Fault Localization , 2017, 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC).

[40]  Wes Masri,et al.  An empirical study of the factors that reduce the effectiveness of coverage-based fault localization , 2009, DEFECTS '09.

[41]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[42]  Qian Yang,et al.  A Survey of Coverage-Based Testing Tools , 2009, Comput. J..

[43]  Zuohua Ding,et al.  Fault localization based on statement frequency , 2016, Inf. Sci..

[44]  Shujuan Jiang,et al.  Cost-effective testing based fault localization with distance based test-suite reduction , 2016, Science China Information Sciences.

[45]  Richard M. Schwartz,et al.  Fast and Robust Neural Network Joint Models for Statistical Machine Translation , 2014, ACL.

[46]  Mary Lou Soffa,et al.  A methodology for controlling the size of a test suite , 1993, TSEM.

[47]  A. Jefferson Offutt,et al.  Test Oracle Strategies for Model-Based Testing , 2017, IEEE Transactions on Software Engineering.