Investigating the Effectiveness of Peer Code Review in Distributed Software Development

Code review is a potential means of improving software quality. To be effective, it depends on different factors, and many have been investigated in the literature to identify the scenarios in which it adds quality to the final code. However, factors associated with distributed software development, which is becoming increasingly common, have been little explored. Geographic distance can impose additional challenges to the reviewing process. We thus in this paper present the results of a quantitative study of the effectiveness of code review in a distributed software project involving 201 members. We investigate factors that can potentially influence the outcomes of peer code review. Our results show that a high number of changed lines of code tends to increase the review duration with a reduced number of messages, while the number of involved teams, locations, and participant reviewers generally improve reviewer contributions, but with a severe penalty to the duration.

[1]  Jacek Czerwonka,et al.  Code Reviews Do Not Find Bugs. How the Current Code Review Best Practice Slows Us Down , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[2]  Shane McIntosh,et al.  The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects , 2014, MSR 2014.

[3]  Chanchal Kumar Roy,et al.  CORRECT: Code reviewer recommendation at GitHub for Vendasta technologies , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[4]  Bikram Sengupta,et al.  A research agenda for distributed software development , 2006, ICSE.

[5]  Bertrand Meyer Design and Code Reviews in the Age of the Internet , 2008, SEAFOOD.

[6]  Andy Zaidman,et al.  Modern code reviews in open-source projects: which problems do they fix? , 2014, MSR 2014.

[7]  Vipin Balachandran,et al.  Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[8]  Hajimu Iida,et al.  Improving code review effectiveness through reviewer recommendations , 2014, CHASE.

[9]  Michael W. Godfrey,et al.  Investigating technical and non-technical factors influencing modern code review , 2015, Empirical Software Engineering.

[10]  David Lo,et al.  Who should review this change?: Putting text and file location analyses together for more accurate recommendations , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[11]  Christian Bird,et al.  Automatically Recommending Peer Reviewers in Modern Code Review , 2016, IEEE Transactions on Software Engineering.

[12]  Christian Bird,et al.  Characteristics of Useful Code Reviews: An Empirical Study at Microsoft , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[13]  Michael E. Fagan Advances in software inspections , 1986, IEEE Transactions on Software Engineering.

[14]  Naoyasu Ubayashi,et al.  A Study of the Quality-Impacting Practices of Modern Code Review at Sony Mobile , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[15]  Hajimu Iida,et al.  Who should review my code? A file location-based code-reviewer recommendation approach for Modern Code Review , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[16]  Sami Kollanus,et al.  Survey of Software Inspection Research , 2009 .

[17]  Hajimu Iida,et al.  Review participation in modern code review , 2017, Empirical Software Engineering.

[18]  Hajimu Iida,et al.  Investigating Code Review Practices in Defective Files: An Empirical Study of the Qt System , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[19]  Alberto Bacchelli,et al.  Expectations, outcomes, and challenges of modern code review , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[20]  Mark C. Paulk,et al.  The Impact of Design and Code Reviews on Software Quality: An Empirical Study Based on PSP Data , 2009, IEEE Transactions on Software Engineering.

[21]  Ricardo J. Machado,et al.  An approach to improving software inspections performance , 2010, 2010 IEEE International Conference on Software Maintenance.

[22]  Xin Yang,et al.  Social network analysis in open source software peer review , 2014, SIGSOFT FSE.

[23]  Christopher D. Hundhausen,et al.  Talking about code: Integrating pedagogical code reviews into early computing courses , 2013, TOCE.

[24]  Victor R. Basili,et al.  Experimentation in software engineering , 1986, IEEE Transactions on Software Engineering.

[25]  Gail C. Murphy,et al.  Removing stagnation from modern code review , 2016, SPLASH.

[26]  Hajimu Iida,et al.  Revisiting Code Ownership and Its Relationship with Software Quality in the Scope of Modern Code Review , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).