Geographical bias in GitHub : perceptions and reality

Open source development has often been considered to be a level playing field for all developers. But there has been little work to investigate if bias plays any role in getting contributions accepted. The work presented in this study tries to understand the influence of geographical location on the evaluation of pull requests in GitHub one of the primary open source development platforms. Using a mixed-methods approach that analyzes 70,000+ pull requests and 2,500+ survey responses, we find that geographical location explains statistically significant differences in pull request acceptance decisions. Compared to the United States, submitters from United Kingdom (22%), Canada (25%), Japan (40%), Netherlands (43%), and Switzerland (58%) have higher chances of getting their pull requests accepted. However, submitters from Germany (15%), Brazil (17%), China (24%), and Italy (19%) have lower chances of getting their pull requests accepted compared to the United States. The probability of pull request acceptance decisions increase by 19% when the submitter and integrator are from the same geographical location. Survey responses from submitters indicate the perception of bias is strong in Brazil and Italy matching our results. Also, 8 out of every 10 integrators feel that it is easy to work with submitters from the same geographical location.

[1]  James D. Herbsleb,et al.  Influence of social and technical factors for evaluating contribution in GitHub , 2014, ICSE.

[2]  Premkumar T. Devanbu,et al.  Open Borders? Immigration in Open Source Projects , 2007, Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007).

[3]  Emerson R. Murphy-Hill,et al.  Gender bias in open source: Pull request acceptance of women versus men , 2016, PeerJ Prepr..

[4]  Georgios Gousios,et al.  Work practices and challenges in pull-based development: the contributor's perspective , 2015, ICSE.

[5]  S. Saha,et al.  Racial and ethnic differences in patient perceptions of bias and cultural competence in health care , 2004, Journal of general internal medicine.

[6]  A. Furnham.  Response bias, social desirability and dissimulation , 1986 .

[7]  Premkumar T. Devanbu,et al.  A large scale study of programming languages and code quality in github , 2014, SIGSOFT FSE.

[8]  Lee Sigelaman.  Question-Order Effects on Presidential Popularity , 1981 .

[9]  James D. Herbsleb,et al.  Impression formation in online peer production: activity traces and personal profiles in github , 2013, CSCW.

[10]  Daniela E. Damian,et al.  Selecting Empirical Methods for Software Engineering Research , 2008, Guide to Advanced Empirical Software Engineering.

[11]  Stephan Diehl,et al.  Small patches get in! , 2008, MSR '08.

[12]  Premkumar T. Devanbu,et al.  Gender and Tenure Diversity in GitHub Teams , 2015, CHI.

[13]  D. Byrne.  The Attraction Paradigm , 1971 .

[14]  Olof Åslund,et al.  Do Anonymous Job Application Procedures Level the Playing Field? , 2012 .

[15]  Anna Sandberg,et al.  Competing biases : Effects of gender and nationality in sports judging , 2015 .

[16]  W. Scacchi.  Free/Open Source Software Development: Recent Research Results and Methods , 2007, Adv. Comput..

[17]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[18]  Cecilia,et al.  Are Emily and Greg More Employable Than Lakisha and Jamal ? A Field Experiment on Labor Market Discrimination , 2007 .

[19]  Sean P. Goggins,et al.  Performance and participation in open source software on GitHub , 2013, CHI Extended Abstracts.

[20]  S. K. Horwitz,et al.  The Effects of Team Diversity on Team Outcomes: A Meta-Analytic Review of Team Demography , 2007 .

[21]  M. Kraska-MIller,et al.  Nonparametric Statistics for Social and Behavioral Sciences , 2013 .

[22]  Georgios Gousios,et al.  A dataset for pull-based development research , 2014, MSR 2014.

[23]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[24]  Q. Vuong.  Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses , 1989 .

[25]  Leif Singer,et al.  Creating a shared understanding of testing culture on a social coding site , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[26]  Ian Alexander,et al.  An introduction to qualitative research , 2000, Eur. J. Inf. Syst..

[27]  Karen A. Jehn,et al.  A review of perceived diversity in teams: Does how members perceive their team's composition affect team processes and outcomes? , 2016 .

[28]  Alexander Serebrenik,et al.  Perceptions of Diversity on Git Hub: A User Survey , 2015, 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering.

[29]  L. Ross,et al.  The Bias Blind Spot: Perceptions of Bias in Self Versus Others , 2002 .

[30]  Arie van Deursen,et al.  An exploratory study of the pull-based software development model , 2014, ICSE.