Modern Code Review: A Case Study at Google

Employing lightweight, tool-based code review of code changes (aka modern code review) has become the norm for a wide variety of open-source and industrial systems. In this paper, we make an exploratory investigation of modern code review at Google. Google introduced code review early on and evolved it over the years; our study sheds light on why Google introduced this practice and analyzes its current status, after the process has been refined through decades of code changes and millions of code reviews. By means of 12 interviews, a survey with 44 respondents, and the analysis of review logs for 9 million reviewed changes, we investigate motivations behind code review at Google, current practices, and developers' satisfaction and challenges.

[1]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[2]  Michael W. Godfrey,et al.  The influence of non-technical factors on code review , 2013, 2013 20th Working Conference on Reverse Engineering (WCRE).

[3]  Ciera Jaspan,et al.  Tricorder: Building a Program Analysis Ecosystem , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[4]  Marcus Ciolkowski,et al.  Conducting on-line surveys in software engineering , 2003, 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings..

[5]  Daniel M. Germán,et al.  Contemporary Peer Review in Action: Lessons from Open Source Development , 2012, IEEE Software.

[6]  Audris Mockus,et al.  Expertise Browser: a quantitative approach to identifying expertise , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[7]  D. Sackett.  Bias in analytic research. , 1979, Journal of chronic diseases.

[8]  Daniel M. Germán,et al.  Peer Review on Open-Source Software Projects: Parameters, Statistical Models, and Theory , 2014, TSEM.

[9]  Shuvendu K. Lahiri,et al.  Helping Developers Help Themselves: Automatic Decomposition of Code Review Changesets , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[10]  Jacek Czerwonka,et al.  Code Reviews Do Not Find Bugs. How the Current Code Review Best Practice Slows Us Down , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[11]  Georgios Gousios,et al.  Untangling fine-grained code changes , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[12]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[13]  Jeffrey C. Carver,et al.  Impact of Peer Code Review on Peer Impression Formation: A Survey , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[15]  W. Thomas,et al.  The Child in America: Behavior Problems and Programs , 2017 .

[16]  Yuki Ueda,et al.  The Impact of a Low Level of Agreement Among Reviewers in a Code Review Process , 2016, OSS.

[17]  Forrest Shull,et al.  Inspecting the History of Inspections: An Example of Evidence-Based Technology Diffusion , 2008, IEEE Software.

[18]  Christian Bird,et al.  Convergent contemporary software peer review practices , 2013, ESEC/FSE 2013.

[19]  Georgios Gousios,et al.  Work Practices and Challenges in Pull-Based Development: The Integrator's Perspective , 2014, ICSE.

[20]  Josh Levenberg,et al.  Why Google stores billions of lines of code in a single repository , 2016, Commun. ACM.

[21]  Margaret-Anne Storey,et al.  – Lessons and Recommendations for Closed Source , 2012 .

[22]  Christian Bird,et al.  Automatically Recommending Peer Reviewers in Modern Code Review , 2016, IEEE Transactions on Software Engineering.

[23]  Daniel M. Germán,et al.  Will my patch make it? And how fast? Case study on the Linux kernel , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[24]  Hajimu Iida,et al.  Review participation in modern code review , 2017, Empirical Software Engineering.

[25]  Hajimu Iida,et al.  Revisiting Code Ownership and Its Relationship with Software Quality in the Scope of Modern Code Review , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[26]  Christian Bird,et al.  Characteristics of Useful Code Reviews: An Empirical Study at Microsoft , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[27]  X. Zhang,et al.  Appendix to the Paper : , 2009 .

[28]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[29]  Georgios Gousios,et al.  Work practices and challenges in pull-based development: the contributor's perspective , 2015, ICSE.

[30]  Christian Bird,et al.  Convergent Software Peer Review Practices , 2013 .

[31]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[32]  Audris Mockus,et al.  Developer fluency: achieving true mastery in software projects , 2010, FSE '10.

[33]  Hajimu Iida,et al.  Who should review my code? A file location-based code-reviewer recommendation approach for Modern Code Review , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[34]  Sami Kollanus,et al.  Survey of Software Inspection Research , 2009 .

[35]  A. Frank Ackerman,et al.  Software inspections: an effective verification process , 1989, IEEE Software.

[36]  Harald C. Gall,et al.  Don't touch my code!: examining the effects of ownership on software quality , 2011, ESEC/FSE '11.

[37]  H. Jansen,et al.  The Logic of Qualitative Survey Research and its Position in the Field of Social Research Methods , 2010 .

[38]  Alberto Bacchelli,et al.  Expectations, outcomes, and challenges of modern code review , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[39]  Priscilla J. Fowler,et al.  Software inspections and the industrial production of software , 1984 .

[40]  Gina Venolia,et al.  Can peer code reviews be exploited for later information needs? , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[41]  Vipin Balachandran,et al.  Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[42]  Michael W. Godfrey,et al.  Code Review Quality: How Developers See It , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[43]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..