Reducing the Cost of Android Mutation Testing

Due to the high market share of Android mobile devices, Android apps dominate the global market in terms of users, developers, and app releases. However, the quality of Android apps is a significant problem. Previously, we developed a mutation analysis-based approach to testing Android apps and showed it to be very effective. However, the computational cost of Android mutation testing is very high, possibly limiting its practical use. This paper presents a cost-reduction approach based on identifying redundancy among mutation operators used in Android mutation analysis. Excluding them can reduce cost without affecting the test quality. We consider a mutation operator to be redundant if tests designed to kill other types of mutants can also kill all or most of the mutants of this operator. We conducted an empirical study with selected open source Android apps. The results of our study show that three operators are redundant and can be excluded from Android mutation analysis. We also suggest updating one operator’s implementation to stop generating trivial mutants. Additionally, we identity subsumption relationships among operators so that the operators subsumed by others can be skipped in Android mutation analysis.

[1]  A. Jefferson Offutt,et al.  Designing Deletion Mutation Operators , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[2]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[3]  A. Jefferson Offutt,et al.  Towards mutation analysis of Android apps , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[4]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[5]  A. Jefferson Offutt,et al.  Mutation operators for testing Android apps , 2017, Inf. Softw. Technol..

[6]  A. Jefferson Offutt,et al.  Empirical Evaluation of the Statement Deletion Mutation Operator , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[7]  A. Jefferson Offutt,et al.  Mutation 2000: uniting the orthogonal , 2001 .

[8]  A. Jefferson Offutt,et al.  Establishing Theoretical Minimal Sets of Mutants , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[9]  A. Jefferson Offutt,et al.  Analyzing the validity of selective mutation with dominator mutants , 2016, SIGSOFT FSE.

[10]  A. Jefferson Offutt,et al.  Finding Redundancy in Web Mutation Operators , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[11]  M. Meeker Internet trends 2015 , 2015 .

[12]  A. Jefferson Offutt,et al.  Is Mutation Analysis Effective at Testing Android Apps? , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[13]  Wynne Hsu,et al.  DESIGN OF MUTANT OPERATORS FOR THE C PROGRAMMING LANGUAGE , 2006 .

[14]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.