Test suite effectiveness: an indicator for open source software quality

Nowadays, open source software is playing a big role in several business contexts. Open source systems have moved from just educational projects to mainstream research area. Successful and active open source projects are numbering more than thousands that needs to be tested and their quality level needs to be determined. There are several ways to measure the quality of software systems. Mainly, measuring the ability of test cases to detect and find defects is used. This research focuses on finding a good technique to evaluate the effectiveness of test cases for finding defects in open source systems. Experiments are conducted on six OSS (open source software). The result shows how the effectiveness of test suite of could give an indication about studied systems quality.

[1]  Tibor Gyimóthy,et al.  Beyond code coverage — An approach for test suite assessment and improvement , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[2]  Jonathan Robie,et al.  Editors , 2003 .

[3]  Alex Groce,et al.  Comparing non-adequate test suites using coverage criteria , 2013, ISSTA.

[4]  Gregory DeKoenigsberg.  How Successful Open Source Projects Work, and How and Why to Introduce Students to the Open Source World , 2008, 2008 21st Conference on Software Engineering Education and Training.

[5]  Laura Inozemtseva,et al.  Predicting Test Suite Effectiveness for Java Programs , 2012 .

[6]  Nousheen Hashmi,et al.  Analyzing test case quality with mutation testing approach , 2015, 2015 Science and Information Conference (SAI).

[7]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[8]  Andreas Zeller,et al.  Mutation-Driven Generation of Unit Tests and Oracles , 2010, IEEE Transactions on Software Engineering.

[9]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[10]  Sergio Segura,et al.  Mutation testing on an object-oriented framework: An experience report , 2011, Inf. Softw. Technol..

[11]  Andreas Zeller,et al.  Checked coverage: an indicator for oracle quality , 2013, Softw. Test. Verification Reliab..

[12]  A. Jefferson Offutt,et al.  Mutation analysis using mutant schemata , 1993, ISSTA '93.

[13]  Kevin Crowston,et al.  Free/Libre open-source software development: What we know and what we do not know , 2012, CSUR.

[14]  René Just,et al.  MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[15]  Mark Aberdour.  A people-focused , 2022 .

[16]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[17]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[18]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[19]  David Lo,et al.  Code coverage and test suite effectiveness: Empirical study with real bugs in large systems , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[20]  Mohammed Akour,et al.  Regression Test-Selection Technique Using Component Model Based Modification: Code to Test Traceability , 2016 .

[21]  Riza Sulaiman,et al.  The process of quality assurance under open source software development , 2011, 2011 IEEE Symposium on Computers & Informatics.

[22]  Ramanath Subramanyam,et al.  Free/Libre Open Source Software Development in Developing and Developed Countries: An Exploratory Study , 2006 .

[23]  Mickaël Delahaye,et al.  A Comparison of Mutation Analysis Tools for Java , 2013, 2013 13th International Conference on Quality Software.