The Long Road Ahead: Ongoing Challenges in Contributing to Large OSS Organizations and What to Do

Open source communities hosted in large foundations operate in a complex socio-technical ecosystem, which includes a heterogeneous mix of projects and stakeholders. Previous work has thus far investigated the challenges faced in OSS communities from the point of view of specific stakeholders, primarily at the level of individual projects. None have yet studied the challenges faced within a large, federated open source organization. In this paper, we aim to bridge this gap to identify ongoing challenges contributors face in a mature OSS organization. To do so, we surveyed 624 contributors at the Apache Software Foundation (ASF) and ran 11 semi-structured follow up interviews. We validated our findings through member checking with the interviewees as well as the ASF Diversity and Inclusion (D&I) committee. The contributions of this paper include: (1) an empirically-evidenced conceptual model of the 88 challenges that contributors face in a mature OSS foundation and (2) a set of 48 community-recommended strategies for alleviating these challenges. Our results show that even well-established and mature organizations still face a variety of individual and project-specific challenges and that it is difficult to design a comprehensive set of processes and guidelines to match the needs and expectations of a diverse and large federated community. Our conceptual challenges model and associated strategies to mitigate them can provide guidance to other OSS foundations and projects helping them in building better support processes and tools to create a successful, thriving community of contributors.

[1]  Khai N. Truong,et al.  "Welcome!": social and psychological predictors of volunteer socializers in online communities , 2013, CSCW.

[2]  Margaret M. Burnett,et al.  Open Source Barriers to Entry, Revisited: A Sociotechnical Perspective , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[3]  Sebastian Spaeth,et al.  Carrots and Rainbows: Motivation and Social Practice in Open Source Software Development , 2012, MIS Q..

[4]  Sogol Balali,et al.  Newcomers’ Barriers. . . Is That All? An Analysis of Mentors’ and Newcomers’ Barriers in OSS Projects , 2018, Computer Supported Cooperative Work (CSCW).

[5]  Tommi Mikkonen,et al.  Organizing for openness: six models for developer involvement in hybrid OSS projects , 2018, J. Internet Serv. Appl..

[6]  Kazi Zakia Sultana,et al.  Expressions of Sentiments during Code Reviews: Male vs. Female , 2018, 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[7]  Jeffrey C. Carver,et al.  Understanding the Impressions, Motivations, and Barriers of One Time Code Contributors to FLOSS Projects: A Survey , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[8]  Volker Gruhn,et al.  An exploratory study of contribution barriers experienced by newcomers to open source software projects , 2014, CSI-SE 2014.

[9]  Marco Aurélio Gerosa,et al.  Understanding and Supporting the Choice of an Appropriate Task to Start with in Open Source Software Communities , 2015, 2015 48th Hawaii International Conference on System Sciences.

[10]  Patrick Chang Boon Lee,et al.  The social context of turnover among information technology professionals , 2002, SIGCPR '02.

[11]  Christoph Treude,et al.  Overcoming Open Source Project Entry Barriers with a Portal for Newcomers , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[12]  Kevin Lumbard,et al.  Eight Observations and 24 Research Questions About Open Source Projects , 2018, Proc. ACM Hum. Comput. Interact..

[13]  Igor Steinmacher,et al.  Hidden Figures: Roles and Pathways of Successful OSS Contributors , 2020, Proc. ACM Hum. Comput. Interact..

[14]  Jonathan Grudin,et al.  Groupware and social dynamics: eight challenges for developers , 1994, CACM.

[15]  Christoph Treude,et al.  Let Me In: Guidelines for the Successful Onboarding of Newcomers to Open Source Projects , 2019, IEEE Software.

[16]  Michael J. Prietula,et al.  Open Collaboration for Innovation: Principles and Performance , 2013, Organ. Sci..

[17]  Marco Aurélio Gerosa,et al.  Recommending mentors to software project newcomers , 2012, 2012 Third International Workshop on Recommendation Systems for Software Engineering (RSSE).

[18]  John M. Levine,et al.  Personnel Turnover and Team Performance , 2005 .

[19]  Janice Singer,et al.  Hipikat: a project memory for software development , 2005, IEEE Transactions on Software Engineering.

[20]  Robert E. Kraut,et al.  Predicting Continued Participation in Newsgroups , 2006, J. Comput. Mediat. Commun..

[21]  Julia E. Hoch,et al.  Leading virtual teams: hierarchical leadership, structural supports, and shared team leadership. , 2014, The Journal of applied psychology.

[22]  Cliff Lampe,et al.  Defining, Understanding, and Supporting Open Collaboration , 2013 .

[23]  Anita Sarma,et al.  Which bug should I fix: helping new developers onboard a new project , 2011, CHASE '11.

[24]  Sharan B. Merriam,et al.  Qualitative Research: A Guide to Design and Implementation , 2009 .

[25]  Marco Aurélio Gerosa,et al.  Almost There: A Study on Quasi-Contributors in Open-Source Software Projects , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[26]  A. Garman,et al.  Ready-to-use-simulation: THE HIDDEN COSTS OF EMPLOYEE TURNOVER , 2005 .

[27]  Marcin Sydow,et al.  Diversity of editors and teams versus quality of cooperative work: experiments on wikipedia , 2017, Journal of Intelligent Information Systems.

[28]  Naoyasu Ubayashi,et al.  Magnet or Sticky? Measuring Project Characteristics from the Perspective of Developer Attraction and Retention , 2016, J. Inf. Process..

[29]  Marco Aurélio Gerosa,et al.  How Gender-Biased Tools Shape Newcomer Experiences in OSS Projects , 2020, IEEE Transactions on Software Engineering.

[30]  Audris Mockus,et al.  Organizational volatility and its effects on software defects , 2010, FSE '10.

[31]  John M. Levine,et al.  To stay or leave?: the relationship of emotional and informational support to commitment in online health support groups , 2012, CSCW.

[32]  Marco Aurélio Gerosa,et al.  A systematic literature review on the barriers faced by newcomers to open source software projects , 2015, Inf. Softw. Technol..

[33]  Gail C. Murphy,et al.  Impact of developer turnover on quality in open-source software , 2015, ESEC/SIGSOFT FSE.

[34]  Helen Sharp,et al.  Software Engineers' Perceptions of Factors in Motivation: The Work, People, Obstacles , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[35]  David García,et al.  It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia , 2015, ICWSM.

[36]  Jr. Frederick P. Brooks,et al.  The mythical man-month (anniversary ed.) , 1995 .

[37]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[38]  Bernard J. Jansen,et al.  Four Types of Toxic People: Characterizing Online Users’ Toxicity over Time , 2020, NordiCHI.

[39]  Emerson Murphy-Hill,et al.  Pushback: Characterizing and Detecting Negative Interpersonal Interactions in Code Review , 2020 .

[40]  Nicolas Ducheneaut,et al.  Socialization in an Open Source Software Community: A Socio-Technical Analysis , 2005, Computer Supported Cooperative Work (CSCW).

[41]  Drew Paine,et al.  From The Matrix to a Model of Coordinated Action (MoCA): A Conceptual Framework of and for CSCW , 2015, CSCW.

[42]  Derek J. Croome Creating the Productive Workplace , 1999 .

[43]  Oded Nov,et al.  Exploring motivations for contributing to open source initiatives: The roles of contribution context and personal values , 2008, Comput. Hum. Behav..

[44]  Gede Artha Azriadi Prana,et al.  Including Everyone, Everywhere: Understanding Opportunities and Challenges of Geographic Gender-Inclusion in OSS , 2020, IEEE Transactions on Software Engineering.

[45]  Cliff Lampe,et al.  Follow the (slash) dot: effects of feedback on new members in an online community , 2005, GROUP.

[46]  B. Glaser Theoretical Sensitivity: Advances in the Methodology of Grounded Theory , 1978 .

[47]  Ed H. Chi,et al.  The singularity is not near: slowing growth of Wikipedia , 2009, Int. Sym. Wikis.

[48]  P. Hinds,et al.  CONFLICT & SHARED IDENTITY IN GEOGRAPHICALLY DISTRIBUTED TEAMS. , 2001 .

[49]  John M. Levine,et al.  Impact of personnel turnover on team performance and cognition. , 2004 .

[50]  Philip J. Guo,et al.  Paradise unplugged: identifying barriers for female participation on stack overflow , 2016, SIGSOFT FSE.

[51]  Christoph Treude,et al.  Twenty Years of Open Source Software: From Skepticism to Mainstream , 2019, IEEE Softw..

[52]  Marco Aurélio Gerosa,et al.  Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects , 2015, CSCW.

[53]  Georgios Gousios,et al.  Work practices and challenges in pull-based development: the contributor's perspective , 2015, ICSE.

[54]  Dawn Nafus,et al.  ‘Patches don’t have gender’: What is not open in open source software , 2012, New Media Soc..

[55]  Carlos Jensen,et al.  Joining Free/Open Source Software Communities: An Analysis of Newbies' First Interactions on Project Mailing Lists , 2011, 2011 44th Hawaii International Conference on System Sciences.

[56]  Marco Tulio Valente,et al.  Why modern open source projects fail , 2017, ESEC/SIGSOFT FSE.

[57]  Marco Aurélio Gerosa,et al.  A theory of the engagement in open source projects via summer of code programs , 2020, ESEC/SIGSOFT FSE.

[58]  Georgios Gousios,et al.  Relationship between geographical location and evaluation of developer contributions in github , 2018, ESEM.

[59]  Sogol Balali,et al.  Recommending Tasks to Newcomers in OSS Projects: How Do Mentors Handle It? , 2020, OpenSym.

[60]  Hichang Cho,et al.  Mudslinging and Manners: Unpacking Conflict in Free and Open Source Software , 2015, CSCW.

[61]  James D. Herbsleb,et al.  Social coding in GitHub: transparency and collaboration in an open software repository , 2012, CSCW.

[62]  John Riedl,et al.  Mentoring in Wikipedia: a clash of cultures , 2011, Int. Sym. Wikis.

[63]  Audris Mockus,et al.  Succession: Measuring transfer of code and developer productivity , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[64]  Rory O'Connor,et al.  A software process engineering approach to improving software team productivity using socioeconomic mechanism design , 2011, SOEN.

[65]  Robbie T. Nakatsu,et al.  A comparative study of important risk factors involved in offshore and domestic outsourcing of software development projects: A two-panel Delphi study , 2009, Inf. Manag..

[66]  Tracy Hall,et al.  The impact of staff turnover on software projects: the importance of understanding what makes software practitioners tick , 2008, SIGMIS CPR '08.

[67]  Audris Mockus,et al.  Inflow and Retention in OSS Communities with Commercial Involvement , 2016, ACM Trans. Softw. Eng. Methodol..

[68]  Kevin Crowston,et al.  Information systems success in free and open source software development: theory and measures , 2006, Softw. Process. Improv. Pract..

[69]  Shannon L. Marlow,et al.  Does team communication represent a one-size-fi ts-all approach ? : A meta-analysis of team communication and performance , 2018 .

[70]  Eduardo Graells-Garrido,et al.  Women through the glass ceiling: gender asymmetries in Wikipedia , 2016, EPJ Data Science.

[71]  James D. Herbsleb,et al.  When Cultures Clash: Participation in Open Source Communities and Its Implications For Organizational Commitment , 2011, ICIS.

[72]  Sonali K. Shah Motivation, Governance, and the Viability of Hybrid Forms in Open Source Software Development , 2006, Manag. Sci..

[73]  Chaim Fershtman,et al.  Open source software: Motivation and restrictive licensing , 2007 .

[74]  Eric S. Raymond,et al.  The Cathedral and the Bazaar , 2000 .

[75]  Volker Gruhn,et al.  Patterns for tearing down contribution barriers to FLOSS projects , 2013, 2013 IEEE 12th International Conference on Intelligent Software Methodologies, Tools and Techniques (SoMeT).

[76]  Thomas Hess,et al.  An Empirical Study of Volunteer Members' Perceived Turnover in Open Source Software Projects , 2012, 2012 45th Hawaii International Conference on System Sciences.

[77]  Jonathan Grudin,et al.  Computer-supported cooperative work: history and focus , 1994, Computer.

[78]  Y. Benkler,et al.  Commons‐based Peer Production and Virtue* , 2006 .

[79]  Gregorio Robles,et al.  Developer Turnover in Global, Industrial Open Source Projects: Insights from Applying Survival Analysis , 2017, 2017 IEEE 12th International Conference on Global Software Engineering (ICGSE).

[80]  Emerson R. Murphy-Hill,et al.  Gender bias in open source: Pull request acceptance of women versus men , 2016, PeerJ Prepr..

[81]  Marco Aurélio Gerosa,et al.  Why do newcomers abandon open source software projects? , 2013, 2013 6th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE).

[82]  S. Georgieva,et al.  Best Practices in Project Management Through a Grounded Theory Lens , 2008 .

[83]  Alexander Serebrenik,et al.  A Data Set for Social Diversity Studies of GitHub Teams , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[84]  Neil A. Ernst,et al.  Do as I Do, Not as I Say: Do Contribution Guidelines Match the GitHub Contribution Process? , 2019, 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[85]  Thomas Moser,et al.  Improving Open Source Software Process Quality Based on Defect Data Mining , 2012, SWQD.

[86]  Fabio Palomba,et al.  Characterizing Women (Not) Contributing to Open-Source , 2019, 2019 IEEE/ACM 2nd International Workshop on Gender Equality in Software Engineering (GE).

[87]  R. Katz The Effects of Group Longevity on Project Communication and Performance. , 1982 .

[88]  Pamela Jordan Basics of qualitative research: Grounded theory procedures and techniques , 1994 .

[89]  Félix Cuadrado,et al.  Apache and Eclipse: Comparing Open Source Project Incubators , 2007, IEEE Softw..

[90]  Emerson Murphy-Hill,et al.  Gender differences and bias in open source: pull request acceptance of women versus men , 2017, PeerJ Comput. Sci..

[91]  Saerom Lee,et al.  Governance strategies for open collaboration: Focusing on resource allocation in open source software development organizations , 2017, Int. J. Inf. Manag..

[92]  Brian Fitzgerald,et al.  The Transformation of Open Source Software , 2006, MIS Q..

[93]  Aaron Halfaker,et al.  ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia , 2020, Proc. ACM Hum. Comput. Interact..

[94]  A. Strauss,et al.  Grounded Theory in Practice , 1997 .

[95]  Volker Gruhn,et al.  On the Relationship between Newcomer Motivations and Contribution Barriers in Open Source Projects , 2017, OpenSym.

[96]  Kazi Zakia Sultana,et al.  Diversity and Inclusion in Open Source Software (OSS) Projects: Where Do We Stand? , 2019, 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[97]  Anita Sarma,et al.  The onion patch: migration in open source ecosystems , 2011, ESEC/FSE '11.

[98]  Julian M. Bass,et al.  Employee Retention and Turnover in Global Software Development: Comparing In-House Offshoring and Offshore Outsourcing , 2018, 2018 IEEE/ACM 13th International Conference on Global Software Engineering (ICGSE).

[99]  Audris Mockus,et al.  Developer fluency: achieving true mastery in software projects , 2010, FSE '10.

[100]  Alexander Serebrenik,et al.  Beyond the Code Itself: How Programmers Really Look at Pull Requests , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS).

[101]  G. Littlepage,et al.  Effects of Task Experience and Group Experience on Group Performance, Member Ability, and Recognition of Expertise , 1997 .

[102]  Christoph Treude,et al.  The Shifting Sands of Motivation: Revisiting What Drives Contributors in Open Source , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[103]  A Mani,et al.  A study of FOSS 2013 survey data using clustering techniques , 2016, 2016 IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE).

[104]  John Riedl,et al.  The Impact and Evolution of Group Diversity in Online Open Collaboration , 2016, Manag. Sci..

[105]  Jeffrey C. Carver,et al.  FLOSS Participants' Perceptions About Gender and Inclusiveness: A Survey , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[106]  M. Graef,et al.  Costing child protective services staff turnover. , 2000, Child welfare.

[107]  Gregorio Robles,et al.  Free and open source software development: the end of the teenage years , 2017, Journal of Internet Services and Applications.

[108]  Aaron Halfaker,et al.  Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work , 2011, Int. Sym. Wikis.

[109]  Leif Singer,et al.  How Social and Communication Channels Shape and Challenge a Participatory Culture in Software Development , 2017, IEEE Transactions on Software Engineering.

[110]  Marco Aurélio Gerosa,et al.  More Common Than You Think: An In-depth Study of Casual Contributors , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[111]  Premkumar T. Devanbu,et al.  Gender and Tenure Diversity in GitHub Teams , 2015, CHI.

[112]  Ruqin Ren,et al.  Crowd Diversity and Performance in Wikipedia: The Mediating Effects of Task Conflict and Communication , 2017, CHI.