On the role of data balancing for machine learning-based code smell detection

Code smells can compromise software quality in the long term by inducing technical debt. For this reason, many approaches aimed at identifying these design flaws have been proposed in the last decade. Most of them are based on heuristics in which a set of metrics (e.g., code metrics, process metrics) is used to detect smelly code components. However, these techniques suffer of subjective interpretation, low agreement between detectors, and threshold dependability. To overcome these limitations, previous work applied Machine Learning techniques that can learn from previous datasets without needing any threshold definition. However, more recent work has shown that Machine Learning is not always suitable for code smell detection due to the highly unbalanced nature of the problem. In this study we investigate several approaches able to mitigate data unbalancing issues to understand their impact on ML-based approaches for code smell detection. Our findings highlight a number of limitations and open issues with respect to the usage of data balancing in ML-based code smell detection.

[1]  Jens Dietrich,et al.  Antipattern and Code Smell False Positives: Preliminary Conceptualization and Classification , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[2]  Yann-Gaël Guéhéneuc,et al.  DECOR: A Method for the Specification and Detection of Code and Design Smells , 2010, IEEE Transactions on Software Engineering.

[3]  Alexander Chatzigeorgiou,et al.  Investigating the Evolution of Bad Smells in Object-Oriented Code , 2010, 2010 Seventh International Conference on the Quality of Information and Communications Technology.

[4]  Andy Zaidman,et al.  Does Refactoring of Test Smells Induce Fixing Flaky Tests? , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[5]  Mauricio A. Saca Refactoring improving the design of existing code , 2017, 2017 IEEE 37th Central America and Panama Convention (CONCAPAN XXXVII).

[6]  Davide Taibi,et al.  How developers perceive smells in source code: A replicated study , 2017, Inf. Softw. Technol..

[7]  Mika Mäntylä,et al.  Code Smell Detection: Towards a Machine Learning-Based Approach , 2013, 2013 IEEE International Conference on Software Maintenance.

[8]  Baldoino Fonseca dos Santos Neto,et al.  Experience report: Evaluating the effectiveness of decision trees for detecting code smells , 2015, 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE).

[9]  Gabriele Bavota,et al.  Anti-Pattern Detection: Methods, Challenges, and Open Issues , 2015, Adv. Comput..

[10]  Gabriele Bavota,et al.  When and Why Your Code Starts to Smell Bad (and Whether the Smells Go Away) , 2015, IEEE Transactions on Software Engineering.

[11]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[12]  Foutse Khomh,et al.  Numerical Signatures of Antipatterns: An Approach Based on B-Splines , 2010, 2010 14th European Conference on Software Maintenance and Reengineering.

[13]  Gabriele Bavota,et al.  An empirical investigation into the nature of test smells , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[14]  Mika Mäntylä,et al.  Comparing and experimenting machine learning techniques for code smell detection , 2015, Empirical Software Engineering.

[15]  M. Mäntylä,et al.  Subjective evaluation of software evolvability using code smells: An empirical study , 2006, Empirical Software Engineering.

[16]  Foutse Khomh,et al.  BDTEX: A GQM-based Bayesian approach for the detection of antipatterns , 2011, J. Syst. Softw..

[17]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[18]  Daniela Cruzes,et al.  The evolution and impact of code smells: A case study of two open source systems , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[19]  Eduardo Figueiredo,et al.  A review-based comparative study of bad smell detection tools , 2016, EASE.

[20]  Foutse Khomh,et al.  IDS: An Immune-Inspired Approach for the Detection of Software Design Smells , 2010, 2010 Seventh International Conference on the Quality of Information and Communications Technology.

[21]  Andy Zaidman,et al.  RETRACTED ARTICLE: The smell of fear: on the relation between test smells and flaky tests , 2019, Empirical Software Engineering.

[22]  Andrea De Lucia,et al.  Comparing Heuristic and Machine Learning Approaches for Metric-Based Code Smell Detection , 2019, 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC).

[23]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[24]  Ian H. Witten,et al.  One-Class Classification by Combining Density and Class Probability Estimation , 2008, ECML/PKDD.

[25]  Francesca Arcelli Fontana,et al.  Automatic detection of bad smells in code: An experimental assessment , 2012, J. Object Technol..

[26]  Gabriele Bavota,et al.  On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation , 2018, Empirical Software Engineering.

[27]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[28]  Francesca Arcelli Fontana,et al.  Code smell severity classification using machine learning techniques , 2017, Knowl. Based Syst..

[29]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[30]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[31]  Ward Cunningham,et al.  The WyCash portfolio management system , 1992, OOPSLA '92.

[32]  Giuliano Antoniol,et al.  Recovering Traceability Links between Code and Documentation , 2002, IEEE Trans. Software Eng..

[33]  Jochen Kreimer,et al.  Adaptive Detection of Design Flaws , 2005, LDTA@ETAPS.

[34]  Tracy Hall,et al.  Code Bad Smells: a review of current knowledge , 2011, J. Softw. Maintenance Res. Pract..

[35]  Aiko Fallas Yamashita,et al.  Do code smells reflect important maintainability aspects? , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[36]  Gabriele Bavota,et al.  Do They Really Smell Bad? A Study on Developers' Perception of Bad Code Smells , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[37]  Gabriele Bavota,et al.  A large-scale empirical study on the lifecycle of code smell co-occurrences , 2018, Inf. Softw. Technol..

[38]  Andrea De Lucia,et al.  Detecting code smells using machine learning techniques: Are we there yet? , 2018, 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[39]  Lin Shi,et al.  Machine learning techniques for code smell detection: A systematic literature review and meta-analysis , 2019, Inf. Softw. Technol..

[40]  Dimitris Kanellopoulos,et al.  Handling imbalanced datasets: A review , 2006 .

[41]  Yann-Gaël Guéhéneuc,et al.  SMURF: A SVM-based Incremental Anti-pattern Detection Approach , 2012, 2012 19th Working Conference on Reverse Engineering.

[42]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[43]  Marcelo de Almeida Maia,et al.  A Systematic Literature Review on Bad Smells–5 W's: Which, When, What, Who, Where , 2018, IEEE Transactions on Software Engineering.

[44]  Foutse Khomh,et al.  An exploratory study of the impact of antipatterns on class change- and fault-proneness , 2011, Empirical Software Engineering.

[45]  Ricardo Baeza-Yates,et al.  Modern Information Retrieval - the concepts and technology behind search, Second edition , 2011 .

[46]  Foutse Khomh,et al.  Tracking Design Smells: Lessons from a Study of God Classes , 2009, 2009 16th Working Conference on Reverse Engineering.

[47]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.