How does code style inconsistency affect pull request integration? An exploratory study on 117 GitHub projects

GitHub is a popular code platform that provides infrastructures to facilitate collaborative development. A Pull Request (PR) is one of the key ideas to support collaboration. Developers are encouraged to submit PRs to ask for the integration of their contributions. In practice, not all submitted PRs can be integrated into the codebase by project maintainers. Existing studies have investigated factors affecting PR integration. Nevertheless, the code style of PRs, which is largely considered by project maintainers, has not been deeply studied yet. In this paper, we performed an exploratory analysis on the effect of code style on PR integration in GitHub. We modeled the code style via the inconsistency between a submitted PR and the existing code in its target codebase. Such modeling makes our study not limited by a specific definition of code style. We conducted our experiments on 50,092 closed PRs in 117 Java projects. Our findings show that: (1) There indeed exists code style inconsistency between PRs and the codebase. (2) Several code style criteria on how to use spaces or indents, make comments, and write code lines with a suitable length, tend to show more inconsistency among PRs. (3) A PR that is consistent with the current code style tends to be merged into the codebase more easily. (4) A PR that violates the current code style is likely to take more time to get closed. Our study shows evidence to developers about how to deliver better contributions to facilitate efficient collaboration.

[1]  Gang Yin,et al.  Reviewer Recommender of Pull-Requests in GitHub , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[2]  Xin Zhang,et al.  How do Multiple Pull Requests Change the Same Code: A Study of Competing Pull Requests in GitHub , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[3]  Ben Shneiderman,et al.  Program indentation and comprehensibility , 1983, CACM.

[4]  James D. Herbsleb,et al.  Let's talk about it: evaluating contributions through discussion in GitHub , 2014, SIGSOFT FSE.

[5]  Rohan Padhye,et al.  A study of external community contribution to open-source projects on GitHub , 2014, MSR 2014.

[6]  Scott W. Ambler,et al.  The Elements of Java™ Style: Index , 2000 .

[7]  Georgios Gousios,et al.  Work practices and challenges in pull-based development: the contributor's perspective , 2015, ICSE.

[8]  Donald Hedeker,et al.  A Practical Guide to Calculating Cohen’s f2, a Measure of Local Effect Size, from PROC MIXED , 2012, Front. Psychology.

[9]  David Marca,et al.  Some Pascal style guidelines , 1981, SIGP.

[10]  Jacob Cohen,et al.  Statistical Power Analysis For The Behavioral Sciences Revised Edition , 1987 .

[11]  Margaret-Anne D. Storey,et al.  Understanding broadcast based peer review on open source software projects , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[12]  Charles A. Sutton,et al.  Learning natural coding conventions , 2014, SIGSOFT FSE.

[13]  Daniela E. Damian,et al.  The promises and perils of mining GitHub , 2009, MSR 2014.

[14]  Robert Kabacoff,et al.  R in Action: Data Analysis and Graphics with R , 2015 .

[15]  Yijun Yu,et al.  Relating Identifier Naming Flaws and Code Quality: An Empirical Study , 2009, 2009 16th Working Conference on Reverse Engineering.

[16]  Paul C. Johnson.  Extension of Nakagawa & Schielzeth's R2GLMM to random slopes models , 2014, Methods in ecology and evolution.

[17]  Leonardo Gresta Paulino Murta,et al.  Developers assignment for analyzing pull requests , 2015, SAC.

[18]  Herb Sutter,et al.  C++ coding standards , 2004 .

[19]  R. E. Berry,et al.  A style analysis of C programs , 1985, CACM.

[20]  Arie van Deursen,et al.  An exploratory study of the pull-based software development model , 2014, ICSE.

[21]  Curtis R. Cook,et al.  A taxonomy for programming style , 1990, CSC '90.

[22]  Ariel M. Cohen-Goldberg,et al.  Phonological Competition within the Word: Evidence from the Phoneme Similarity Effect in Spoken Production. , 2012 .

[23]  Chanchal Kumar Roy,et al.  An insight into the pull requests of GitHub , 2014, MSR 2014.

[24]  Premkumar T. Devanbu,et al.  Wait for It: Determinants of Pull Request Evaluation Latency on GitHub , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[25]  Eirini Kalliamvakou,et al.  Open Source-Style Collaborative Development Practices in Commercial Projects Using GitHub , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[26]  Leon Moonen,et al.  Evaluating the relation between coding standard violations and faultswithin and across software versions , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[27]  Shinichi Nakagawa,et al.  A general and simple method for obtaining R2 from generalized linear mixed‐effects models , 2013 .

[28]  Ciera Jaspan,et al.  Lessons from building static analysis tools at Google , 2018, Commun. ACM.

[29]  Steve McConnell,et al.  Code complete - a practical handbook of software construction, 2nd Edition , 1993 .

[30]  J. Grainger,et al.  Native language influences on word recognition in a second language: a megastudy. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[31]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[32]  Alberto Bacchelli,et al.  Expectations, outcomes, and challenges of modern code review , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[33]  Leonardo Gresta Paulino Murta,et al.  Acceptance factors of pull requests in open-source projects , 2015, SAC.

[34]  Vipin Balachandran,et al.  Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[35]  Gang Yin,et al.  Investigating social media in GitHub's pull-requests: a case study on Ruby on Rails , 2014, CrowdSoft 2014.

[36]  Premkumar T. Devanbu,et al.  Quality and productivity outcomes relating to continuous integration in GitHub , 2015, ESEC/SIGSOFT FSE.

[37]  Eleni Stroulia,et al.  Code convention adherence in evolving software , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[38]  Premkumar T. Devanbu,et al.  Will They Like This? Evaluating Code Contributions with Language Models , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[39]  Jacob Cohen.  Preface to the Revised Edition , 1977 .

[40]  Georgios Gousios,et al.  Automatically Prioritizing Pull Requests , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[41]  M. Graham.  CONFRONTING MULTICOLLINEARITY IN ECOLOGICAL MULTIPLE REGRESSION , 2003 .

[42]  Mika Mäntylä,et al.  What Types of Defects Are Really Discovered in Code Reviews? , 2009, IEEE Transactions on Software Engineering.

[43]  Michael J. Rees,et al.  Automatic assessment aids for Pascal programs , 1982, SIGP.

[44]  Jan Hauke,et al.  Comparison of Values of Pearson's and Spearman's Correlation Coefficients on the Same Sets of Data , 2011 .

[45]  James D. Herbsleb,et al.  Influence of social and technical factors for evaluating contribution in GitHub , 2014, ICSE.

[46]  Christoph Treude,et al.  AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).