Improving Code Review by Predicting Reviewers and Acceptance of Patches

Code reviews are an important part of software development because they help to increase code quality and reliability. For this paper, we observed the review processes of two open-source projects, Firefox and Mozilla Core. We noticed that code reviews are mostly organized manually. In particular, finding appropriate reviewers is a complex and time-consuming task and, surprisingly, impacts the review outcome: review requests without an initial reviewer assignment have lower chances to be accepted (and take longer). Based on our observations we propose two improvements: (1) predict whether a given patch is acceptable and (2) suggest reviewers for a patch. We implemented and tested both approaches for the Firefox and Mozilla Core projects. In our experiments, the prediction accuracy was 73% for the the review outcome and 51‐80% for the reviewer recommendation. The values for accuracy are higher than those of comparable approaches and are high enough to be useful and applicable in practice.

[1]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[2]  Andreas Zeller,et al.  Predicting faults from cached history , 2008, ISEC '08.

[3]  Nachiappan Nagappan,et al.  Predicting defects using network analysis on dependency graphs , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[4]  Justin P. Johnson,et al.  Collaboration, Peer Review and Open Source Software , 2004, Inf. Econ. Policy.

[5]  Gail C. Murphy,et al.  Who should fix this bug? , 2006, ICSE.

[6]  Daniel M. German,et al.  Open source software peer review practices , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[7]  Mika Mäntylä,et al.  What Types of Defects Are Really Discovered in Code Reviews? , 2009, IEEE Transactions on Software Engineering.

[8]  M. Uihlein.  Open , 2018 .

[9]  Stephane Lussier,et al.  New tricks: how open source changed the way my team works , 2004, IEEE Software.

[10]  Robert E. Cole,et al.  From a Firm-Based to a Community-Based Model of Knowledge Creation: The Case of the Linux Kernel Development , 2003, Organ. Sci..

[11]  Zhendong Su,et al.  DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones , 2007, 29th International Conference on Software Engineering (ICSE'07).

[12]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[13]  Premkumar T. Devanbu,et al.  Detecting Patch Submission and Acceptance in OSS Projects , 2007, Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007).

[14]  Stephan Diehl,et al.  Small patches get in! , 2008, MSR '08.

[15]  Shahedul Huq Khandkar,et al.  The role of patch review in software evolution: an analysis of the mozilla firefox , 2009, IWPSE-Evol '09.

[16]  Harvey P. Siy,et al.  Understanding the Effects of Developer Activities on Inspection Interval , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[17]  Thomas Zimmermann,et al.  Duplicate bug reports considered harmful … really? , 2008, 2008 IEEE International Conference on Software Maintenance.

[18]  Adam A. Porter,et al.  Reducing Inspection Interval in Large-Scale Software Development , 2002, IEEE Trans. Software Eng..

[19]  Stefan Biffl,et al.  An empirical study on groupware support for software inspection meetings , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[20]  Laurence Brothers,et al.  ICICLE: groupware for code inspection , 1990, CSCW '90.

[21]  Audris Mockus,et al.  Understanding the sources of variation in software inspections , 1998, TSEM.

[22]  Ramon C. Littell,et al.  Exact Statistical Methods for Data Analysis , 2001, Technometrics.

[23]  Stan Matwin,et al.  Feature Engineering for Text Classification , 1999, ICML.

[24]  Gina Venolia,et al.  Can peer code reviews be exploited for later information needs? , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[25]  Vahid Mashayekhi,et al.  CAIS: collaborative asynchronous inspection of software , 1994, SIGSOFT '94.

[26]  Chris Sauer,et al.  Technical Reviews: A Behaviorally Motivated Program of Research , 2022 .

[27]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[28]  O. Laitenberger.  A Survey of Software Inspection Technologies , 2001 .

[29]  Elaine J. Weyuker,et al.  Does calling structure information improve the accuracy of fault prediction? , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[30]  Paul Anderson,et al.  Tool Support for Fine-Grained Software Inspection , 2003, IEEE Softw..

[31]  Thomas Zimmermann,et al.  Improving bug triage with bug tossing graphs , 2009, ESEC/FSE '09.

[32]  A. Frank Ackerman,et al.  Software inspections: an effective verification process , 1989, IEEE Software.

[33]  Jai Asundi,et al.  Patch Review Processes in Open Source Software Development Communities: A Comparative Case Study , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[34]  Priscilla J. Fowler,et al.  Software inspections and the industrial production of software , 1984 .

[35]  Yi Zhang,et al.  Classifying Software Changes: Clean or Buggy? , 2008, IEEE Transactions on Software Engineering.

[36]  Lawrence G. Votta,et al.  Does every inspection need a meeting? , 1993, SIGSOFT '93.

[37]  Harvey P. Siy,et al.  An experiment to assess the cost-benefits of code inspections in large scale software development , 1995, SIGSOFT '95.

[38]  Gérard Memmi,et al.  Scrutiny: A Collaborative Inspection and Review System , 1993, ESEC.

[39]  James M. Bieman,et al.  The FreeBSD project: a replication case study of open source development , 2005, IEEE Transactions on Software Engineering.

[40]  Christopher Podmore.  Information economics and policy: In the United States edited by Michael Rubin Libraries Unlimited, Littleton, CO 1983, 340 pp , 1984 .

[41]  Vahid Mashayekhi,et al.  A Case Study of Distributed, Asynchronous Software Inspection , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[42]  J. David Morgenthaler,et al.  Using FindBugs on production software , 2007, OOPSLA '07.

[43]  Audris Mockus,et al.  Succession: Measuring transfer of code and developer productivity , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[44]  Ethem Alpaydin,et al.  Introduction to Machine Learning (Adaptive Computation and Machine Learning) , 2004 .

[45]  Thomas D. LaToza,et al.  Maintaining mental models: a study of developer work habits , 2006, ICSE.