Are you smelling it? Investigating how similar developers detect code smells

Abstract Context A code smell indicates a poor implementation choice that often worsens software quality. Thus, code smell detection is an elementary technique to identify refactoring opportunities in software systems. Unfortunately, there is limited knowledge on how similar two or more developers detect smells in code. In particular, few studies have investigated if developers agree or disagree when recognizing a smell and which factors can influence on such (dis)agreement. Objective We perform a broader study to investigate how similar the developers detect code smells. We also analyze whether certain factors related to the developers’ profiles concerning background and experience may influence such (dis)agreement. Moreover, we analyze if the heuristics adopted by developers on detecting code smells may influence on their (dis)agreement. Method We conducted an empirical study with 75 developers who evaluated instances of 15 different code smell types. For each smell type, we analyzed the agreement among the developers and we assessed the influence of 6 different factors on the developers’ evaluations. Altogether more than 2700 evaluations were collected, resulting in substantial quantitative and qualitative analyses. Results The results indicate that the developers presented a low agreement on detecting all 15 smell types analyzed in our study. The results also suggest that factors related to background and experience did not have a consistent influence on the agreement among the developers. On the other hand, the results show that the agreement was consistently influenced by specific heuristics employed by developers. Conclusions Our findings reveal that the developers detect code smells in significantly different ways. As a consequence, these findings introduce some questions concerning the results of previous studies that did not consider the different perceptions of developers on detecting code smells. Moreover, our findings shed light towards improving state-of-the-art techniques for accurate, customized detection of code smells.

[1]  Baldoino Fonseca dos Santos Neto,et al.  Using developers' feedback to improve code smell detection , 2015, SAC.

[2]  Yann-Gaël Guéhéneuc,et al.  DECOR: A Method for the Specification and Detection of Code and Design Smells , 2010, IEEE Transactions on Software Engineering.

[3]  Foutse Khomh,et al.  An Empirical Study of the Impact of Two Antipatterns, Blob and Spaghetti Code, on Program Comprehension , 2011, 2011 15th European Conference on Software Maintenance and Reengineering.

[4]  Nenad Medvidovic,et al.  Are automatically-detected code anomalies relevant to architectural modularity?: an exploratory analysis of evolving systems , 2012, AOSD.

[5]  Alessandro F. Garcia,et al.  Detecting architecturally-relevant code anomalies: a case study of effectiveness and effort , 2014, SAC.

[6]  Foutse Khomh,et al.  A Bayesian Approach for the Detection of Code and Design Smells , 2009, 2009 Ninth International Conference on Quality Software.

[7]  Stéphane Ducasse,et al.  Object-Oriented Metrics in Practice , 2005 .

[8]  Francesca Arcelli Fontana,et al.  Automatic detection of bad smells in code: An experimental assessment , 2012, J. Object Technol..

[9]  Martin Fowler,et al.  Refactoring - Improving the Design of Existing Code , 1999, Addison Wesley object technology series.

[10]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[11]  Francesca Arcelli Fontana,et al.  Investigating the Impact of Code Smells on System's Quality: An Empirical Study on Systems of Different Application Domains , 2013, 2013 IEEE International Conference on Software Maintenance.

[12]  Zhendong Niu,et al.  Facilitating software refactoring with appropriate resolution order of bad smells , 2009, ESEC/FSE '09.

[13]  Gabriele Bavota,et al.  Do They Really Smell Bad? A Study on Developers' Perception of Bad Code Smells , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[14]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[15]  Mika Mäntylä,et al.  An experiment on subjective evolvability evaluation of object-oriented software: explaining factors and interrater agreement , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[16]  Mika Mäntylä,et al.  Comparing and experimenting machine learning techniques for code smell detection , 2015, Empirical Software Engineering.

[17]  Francesca Arcelli Fontana,et al.  An Experience Report on Using Code Smells Detection Tools , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[18]  Radu Marinescu,et al.  Detection strategies: metrics-based rules for detecting design flaws , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[19]  M. Mäntylä,et al.  Subjective evaluation of software evolvability using code smells: An empirical study , 2006, Empirical Software Engineering.

[20]  Jurriaan Hage,et al.  How do professionals perceive legacy systems and software modernization? , 2014, ICSE.

[21]  Alessandro F. Garcia,et al.  On the Relevance of Code Anomalies for Identifying Architecture Degradation Symptoms , 2012, 2012 16th European Conference on Software Maintenance and Reengineering.

[22]  José Amancio M. Santos,et al.  An exploratory study to investigate the impact of conceptualization in god class detection , 2013, EASE '13.

[23]  Alessandro F. Garcia,et al.  Code Anomalies Flock Together: Exploring Code Anomaly Agglomerations for Locating Design Problems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[24]  Ghulam Rasool,et al.  A review of code smell mining techniques , 2015, J. Softw. Evol. Process..

[25]  Forrest Shull,et al.  Building empirical support for automated code smell detection , 2010, ESEM '10.

[26]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[27]  Foutse Khomh,et al.  An exploratory study of the impact of antipatterns on class change- and fault-proneness , 2011, Empirical Software Engineering.

[28]  Gabriele Bavota,et al.  Mining Version Histories for Detecting Code Smells , 2015, IEEE Transactions on Software Engineering.

[29]  Yann-Gaël Guéhéneuc,et al.  SMURF: A SVM-based Incremental Anti-pattern Detection Approach , 2012, 2012 19th Working Conference on Reverse Engineering.

[30]  Alessandro F. Garcia,et al.  Enhancing the Detection of Code Anomalies with Architecture-Sensitive Strategies , 2013, 2013 17th European Conference on Software Maintenance and Reengineering.

[31]  M.J. Munro,et al.  Product Metrics for Automatic Identification of "Bad Smell" Design Problems in Java Source-Code , 2005, 11th IEEE International Software Metrics Symposium (METRICS'05).

[32]  Sven Apel,et al.  Measuring and modeling programming experience , 2013, Empirical Software Engineering.

[33]  Tracy Hall,et al.  Code Bad Smells: a review of current knowledge , 2011, J. Softw. Maintenance Res. Pract..

[34]  Cláudio Sant'Anna,et al.  Identifying Code Smells with Multiple Concern Views , 2010, 2010 Brazilian Symposium on Software Engineering.

[35]  Gabriele Bavota,et al.  Detecting bad smells in source code using change history information , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[36]  Ricardo Terra,et al.  Does Technical Debt Lead to the Rejection of Pull Requests? , 2016, SBSI.

[37]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.