tsDetect: an open source test smells detection tool

The test code, just like production source code, is subject to bad design and programming practices, also known as smells. The presence of test smells in a software project may affect the quality, maintainability, and extendability of test suites making them less effective in finding potential faults and quality issues in the project's production code. In this paper, we introduce tsDetect, an automated test smell detection tool for Java software systems that uses a set of detection rules to locate existing test smells in test code. We evaluate the effectiveness of tsDetect on a benchmark of 65 unit test files containing instances of 19 test smell types. Results show that tsDetect achieves a high detection accuracy with an average precision score of 96% and an average recall score of 97%. tsDetect is publicly available, with a demo video, at: https://testsmells.github.io/

[1]  Gabriele Bavota,et al.  On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation , 2018, Empirical Software Engineering.

[2]  Dong Jae Kim,et al.  An Empirical Study on the Evolution of Test Smell , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion).

[3]  Andy Zaidman,et al.  On the Relation of Test Smells to Software Code Quality , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[4]  Bart Van Rompaey,et al.  TestQ: Exploring Structural and Maintenance Characteristics of Unit Test Suites , 2008 .

[5]  Mohamed Wiem Mkaouer,et al.  On the distribution of test smells in open source Android applications: an exploratory study , 2019, CASCON.

[6]  Vahid Garousi,et al.  TeCReVis: A Tool for Test Coverage and Test Redundancy Visualization , 2010, TAIC PART.

[7]  Einar W. Høst,et al.  Debugging Method Names , 2009, ECOOP.

[8]  Mohamed Wiem Mkaouer,et al.  An Exploratory Study on the Refactoring of Unit Test Files in Android Applications , 2020, ICSE.

[9]  Davide Spadini,et al.  Investigating Developer Perception on Test Smells Using Better Code Hub , 2019, SATToSE.

[10]  Arie van Deursen,et al.  Strategies for avoiding text fixture smells during software evolution , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[11]  Gabriele Bavota,et al.  An empirical investigation into the nature of test smells , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[12]  Helmut Neukirchen,et al.  Utilising Code Smells to Detect Quality Problems in TTCN-3 Test Suites , 2007, TestCom/FATES.

[13]  Gabriele Bavota,et al.  Are test smells really harmful? An empirical study , 2014, Empirical Software Engineering.

[14]  Giuliano Antoniol,et al.  Linguistic antipatterns: what they are and how developers perceive them , 2015, Empirical Software Engineering.

[15]  Gerard Meszaros XUnit test patterns and smells: improving the ROI of test code , 2010, SPLASH/OOPSLA Companion.

[16]  Gerard Meszaros,et al.  xUnit Test Patterns: Refactoring Test Code , 2007 .

[17]  Anthony Peruma,et al.  Modeling the Relationship Between Identifier Name and Behavior , 2019, 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[18]  Mohamed Wiem Mkaouer,et al.  Contextualizing rename decisions using refactorings, commit messages, and data types , 2020, J. Syst. Softw..

[19]  Arie van Deursen,et al.  Automated Detection of Test Fixture Strategies and Smells , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[20]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[21]  Stéphane Ducasse,et al.  Rule-based Assessment of Test Quality , 2007, J. Object Technol..

[22]  Alberto Bacchelli,et al.  Investigating Severity Thresholds for Test Smells , 2020, 2020 IEEE/ACM 17th International Conference on Mining Software Repositories (MSR).

[23]  Michael D. Ernst,et al.  Empirically revisiting the test independence assumption , 2014, ISSTA 2014.

[24]  Andrea De Lucia,et al.  Automatic Test Smell Detection Using Information Retrieval Techniques , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[25]  Mohamed Wiem Mkaouer,et al.  An empirical investigation of how and why developers rename identifiers , 2018, IWoR@ASE.

[26]  Mohamed Wiem Mkaouer,et al.  Contextualizing Rename Decisions using Refactorings and Commit Messages , 2019, 2019 19th International Working Conference on Source Code Analysis and Manipulation (SCAM).