Why is my code change abandoned?

Abstract Context: Software developers contribute numerous changes every day to the code review systems. However, not all submitted changes are merged into a codebase because they might not pass the code review process. Some changes would be abandoned or be asked for resubmission after improvement, which results in more workload for developers and reviewers, and more delays to deliverables. Objective: To understand the underlying reasons why changes are abandoned, we conduct an empirical study on the code review of four open source projects (Eclipse, LibreOffice, OpenStack, and Qt). Method: First, we manually analyzed 1459 abandoned changes. Second, we leveraged the open card sorting method to label these changes with reasons why they were abandoned, and we identified 12 categories of reasons. Next, we further investigated the frequency distribution of the categories across projects. Finally, we studied the relationship between the categories and time-to-abandonment. Results: Our findings include the following: (1) Duplicate changes are the majority of the abandoned changes; (2) the frequency distribution of abandoned changes across the 12 categories is similar for the four open source projects; (3) 98.39% of the changes are abandoned within a year. Conclusion: Our study concluded the root causes of abandoned changes, which will help developers submit high-quality code changes.

[1]  Michael W. Godfrey,et al.  Investigating technical and non-technical factors influencing modern code review , 2015, Empirical Software Engineering.

[2]  Bo Guo,et al.  Interactively Decomposing Composite Changes to Support Code Review and Regression Testing , 2017, 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC).

[3]  Naoyasu Ubayashi,et al.  A Study of the Quality-Impacting Practices of Modern Code Review at Sony Mobile , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[4]  Collin McMillan,et al.  Towards Automatic Generation of Short Summaries of Commits , 2017, 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC).

[5]  Jesús M. González-Barahona,et al.  Analyzing Gerrit Code Review Parameters with Bicho , 2014, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[6]  Michael W. Godfrey,et al.  Code Review Quality: How Developers See It , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[7]  DongGyun Han,et al.  Writing Acceptable Patches: An Empirical Study of Open Source Project Patches , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[8]  Roel Wieringa,et al.  Harmfulness of Code Duplication - A Structured Review of the Evidence , 2009, EASE.

[9]  Christian Bird,et al.  Convergent contemporary software peer review practices , 2013, ESEC/FSE 2013.

[10]  Daniel M. Germán,et al.  Will my patch make it? And how fast? Case study on the Linux kernel , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[11]  Forrest Shull,et al.  Inspecting the History of Inspections: An Example of Evidence-Based Technology Diffusion , 2008, IEEE Software.

[12]  Zhenchang Xing,et al.  Neural-Machine-Translation-Based Commit Message Generation: How Far Are We? , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[13]  Priscilla J. Fowler,et al.  Software inspections and the industrial production of software , 1984 .

[14]  Philip M. Johnson Reengineering inspection , 1998, CACM.

[15]  Ramanath Subramanyam,et al.  On The Value of Code Inspections for Software Project Management: An Empirical Analysis , 2006, AMCIS.

[16]  Mario Linares Vásquez,et al.  ChangeScribe: A Tool for Automatically Generating Commit Messages , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[17]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[18]  Lawrence G. Votta,et al.  Does every inspection need a meeting? , 1993, SIGSOFT '93.

[19]  Michael W. Godfrey,et al.  The Secret Life of Patches: A Firefox Case Study , 2012, 2012 19th Working Conference on Reverse Engineering.

[20]  Gabriele Bavota,et al.  Four eyes are better than two: On the impact of code reviews on software quality , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[21]  Christian Bird,et al.  Gerrit software code review data from Android , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[22]  Tsvi Kuflik,et al.  Estimating the Required Code Inspection Team Size , 2007, IEEE International Conference on Software-Science, Technology & Engineering (SwSTE'07).

[23]  Georgios Gousios,et al.  A Mixed Methods Approach to Mining Code Review Data , 2015, The Art and Science of Analyzing Software Data.

[24]  Christian Bird,et al.  Characteristics of Useful Code Reviews: An Empirical Study at Microsoft , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[25]  Margaret-Anne D. Storey,et al.  Understanding broadcast based peer review on open source software projects , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[26]  Thomas Zimmermann,et al.  Improving Code Review by Predicting Reviewers and Acceptance of Patches , 2009 .

[27]  David Lo,et al.  Early prediction of merged code changes to prioritize reviewing tasks , 2018, Empirical Software Engineering.

[28]  Marc Roper,et al.  Practical Code Inspection Techniques for Object-Oriented Systems: An Experimental Comparison , 2003, IEEE Softw..

[29]  David Davenport,et al.  Impact of maintainability defects on code inspections , 2010, ESEM '10.

[30]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[31]  Daniel M. German,et al.  Open source software peer review practices , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[32]  Collin McMillan,et al.  Automatically generating commit messages from diffs using neural machine translation , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[33]  Shane McIntosh,et al.  An empirical study of the impact of modern code review practices on software quality , 2015, Empirical Software Engineering.

[34]  Hajimu Iida,et al.  Investigating Code Review Practices in Defective Files: An Empirical Study of the Qt System , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[35]  Kiminao Kogiso,et al.  Code Review Participation: Game Theoretical Modeling of Reviewers in Gerrit Datasets , 2016, 2016 IEEE/ACM Cooperative and Human Aspects of Software Engineering (CHASE).

[36]  Alberto Bacchelli,et al.  Expectations, outcomes, and challenges of modern code review , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[37]  Vipin Balachandran,et al.  Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[38]  Victor R. Basili,et al.  An Empirical Study of Communication in Code Inspections , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[39]  Nicole Novielli,et al.  Confusion Detection in Code Reviews , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[40]  Hajimu Iida,et al.  Review participation in modern code review , 2017, Empirical Software Engineering.

[41]  Kenichi Matsumoto,et al.  The impact of human factors on the participation decision of reviewers in modern code review , 2018, Empirical Software Engineering.

[42]  Roberto Almeida Bittencourt,et al.  Patch rejection in Firefox: negative reviews, backouts, and issue reopening , 2015, Journal of Software Engineering Research and Development.

[43]  Hajimu Iida,et al.  Who should review my code? A file location-based code-reviewer recommendation approach for Modern Code Review , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[44]  A. Frank Ackerman,et al.  Software inspections: an effective verification process , 1989, IEEE Software.

[45]  Michael W. Godfrey,et al.  The influence of non-technical factors on code review , 2013, 2013 20th Working Conference on Reverse Engineering (WCRE).

[46]  Michael W. Godfrey,et al.  Investigating code review quality: Do people and participation matter? , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[47]  Shane McIntosh,et al.  The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects , 2014, MSR 2014.

[48]  Mario Linares Vásquez,et al.  On Automatically Generating Commit Messages via Summarization of Source Code Changes , 2014, 2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation.

[49]  Shuvendu K. Lahiri,et al.  Helping Developers Help Themselves: Automatic Decomposition of Code Review Changesets , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[50]  Michael Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[51]  Stephan Diehl,et al.  Small patches get in! , 2008, MSR '08.