Search-Based Refactoring: Metrics Are Not Enough

Search-based Software Engineering (SBSE) techniques have been applied extensively to refactor software, often based on metrics that describe the object-oriented structure of an application. Recent work shows that in some cases applying popular SBSE tools to open-source software does not necessarily lead to an improved version of the software as assessed by some subjective criteria. Through a survey of professionals, we investigate the relationship between popular SBSE refactoring metrics and the subjective opinions of software engineers. We find little or no correlation between the two. Through qualitative analysis, we find that a simple static view of software is insufficient to assess software quality, and that software quality is dependent on factors that are not amenable to measurement via metrics. We recommend that future SBSE refactoring research should incorporate information about the dynamic behaviour of software, and conclude that a human-in-the-loop approach may be the only way to refactor software in a manner helpful to an engineer.

[1]  Audris Mockus,et al.  Questioning software maintenance metrics: A comparative case study , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[2]  Rainer Koschke,et al.  Program complexity metrics and programmer opinions , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[3]  Ian C. Parmee,et al.  Elegant Object-Oriented Software Design via Interactive, Evolutionary Computation , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Shawn A. Ross,et al.  Survey Methodology , 2005, The SAGE Encyclopedia of the Sociology of Religion.

[5]  Hürevren Kiliç,et al.  An Empirical Study About Search-Based Refactoring Using Alternative Multiple and Population-Based Search Techniques , 2011, ISCIS.

[6]  Mark Harman,et al.  An empirical study of the robustness of two module clustering fitness functions , 2005, GECCO '05.

[7]  Betty H. C. Cheng,et al.  On the use of genetic programming for automated refactoring and the introduction of design patterns , 2010, GECCO '10.

[8]  Deepak Goyal,et al.  A hierarchical model for object-oriented design quality assessment , 2015 .

[9]  Márcio de Oliveira Barros,et al.  What Can a Big Program Teach Us about Optimization? , 2013, SSBSE.

[10]  Goran Glavas,et al.  Solving the Class Responsibility Assignment Problem Using Metaheuristic Approach , 2011, J. Comput. Inf. Technol..

[11]  Goran Glavas,et al.  Metaheuristic approach to class responsibility assignment problem , 2011, Proceedings of the ITI 2011, 33rd International Conference on Information Technology Interfaces.

[12]  Graham R. Gibbs,et al.  Analyzing Biographies and Narratives , 2014 .

[13]  Guilherme Horta Travassos,et al.  A systematic review of cross- vs. within- company cost estimation studies , 2006 .

[14]  Marouane Kessentini,et al.  On the Use of Machine Learning and Search-Based Software Engineering for Ill-Defined Fitness Function: A Case Study on Software Refactoring , 2014, SSBSE.

[15]  Marian Petre,et al.  UML in practice , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[16]  Graham R. Gibbs,et al.  Analyzing Qualitative Data , 2014 .

[17]  Gabriele Bavota,et al.  An experimental investigation on the innate relationship between quality and refactoring , 2015, J. Syst. Softw..

[18]  Hürevren Kiliç,et al.  A Web-Service for Automated Software Refactoring Using Artificial Bee Colony Optimization , 2012, ICSI.

[19]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[20]  Mark Harman,et al.  Pareto optimal search based refactoring at the design level , 2007, GECCO '07.

[21]  Christopher Alexander,et al.  The Timeless Way of Building , 1979 .

[22]  Mel Ó Cinnéide,et al.  Automated Design Improvement by Example , 2007, SoMeT.

[23]  Harry Hochheiser,et al.  Research Methods for Human-Computer Interaction , 2008 .

[24]  Mark Harman,et al.  Experimental assessment of software metrics using automated refactoring , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[25]  Hedley Apperly,et al.  Service- and component-based development : using Select Perspective and UML , 2003 .

[26]  Graham R. Gibbs,et al.  Analysing Qualitative Data , 2008 .

[27]  Danielle Azar,et al.  A hybrid heuristic approach to optimize rule-based software quality estimation models , 2009, Inf. Softw. Technol..

[28]  Iman Hemati Moghadam,et al.  Code-Imp: a tool for automated search-based refactoring , 2011, WRT '11.

[29]  Ronald Czaja,et al.  Designing Surveys: A Guide to Decisions and Procedures , 1995 .

[30]  John A. Clark,et al.  Metrics are fitness functions too , 2004 .