Socio-technical developer networks: should we trust our measurements?

Software development teams must be properly structured to provide effectiv collaboration to produce quality software. Over the last several years, social network analysis (SNA) has emerged as a popular method for studying the collaboration and organization of people working in large software development teams. Researchers have been modeling networks of developers based on socio-technical connections found in software development artifacts. Using these developer networks, researchers have proposed several SNA metrics that can predict software quality factors and describe the team structure. But do SNA metrics measure what they purport to measure? The objective of this research is to investigate if SNA metrics represent socio-technical relationships by examining if developer networks can be corroborated with developer perceptions. To measure developer perceptions, we developed an online survey that is personalized to each developer of a development team based on that developer's SNA metrics. Developers answered questions about other members of the team, such as identifying their collaborators and the project experts. A total of 124 developers responded to our survey from three popular open source projects: the Linux kernel, the PHP programming language, and the Wireshark network protocol analyzer. Our results indicate that connections in the developer network are statistically associated with the collaborators whom the developers named. Our results substantiate that SNA metrics represent socio-technical relationships in open source development projects, while also clarifying how the developer network can be interpreted by researchers and practitioners.

[1]  Laurie A. Williams,et al.  Secure open source collaboration: an empirical study of linus' law , 2009, CCS.

[2]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[3]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[4]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[5]  Brendan Murphy,et al.  Can developer-module networks predict failures? , 2008, SIGSOFT '08/FSE-16.

[6]  Michael Gertz,et al.  Mining email social networks in Postgres , 2006, MSR '06.

[7]  Laurie A. Williams,et al.  Strengthening the empirical analysis of the relationship between Linus' Law and software security , 2010, ESEM '10.

[8]  Daniela E. Damian,et al.  Predicting build failures using social network analysis on developer communication , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[9]  Ulrik Brandes,et al.  Network Analysis: Methodological Foundations (Lecture Notes in Computer Science) , 2005 .

[10]  Victor R. Basili,et al.  The influence of organizational structure on software quality , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[11]  Laurie A. Williams,et al.  Predicting failures with developer networks and social network analysis , 2008, SIGSOFT '08/FSE-16.

[12]  Harald C. Gall,et al.  Does distributed development affect software quality? An empirical case study of Windows Vista , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[13]  Anita Sarma,et al.  Tesseract: Interactive visual exploration of socio-technical relationships in software development , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[14]  Premkumar T. Devanbu,et al.  Validity of network analyses in Open Source Projects , 2010, 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010).

[15]  Eric Lease Morgan,et al.  Review of The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary by Eric S. Raymond, Sebastopol, Calif.: O'Reilly, 1999 , 2000 .

[16]  Chris Arney.  Network Analysis: Methodological Foundations , 2012 .

[17]  E. Trist,et al.  Some Social and Psychological Consequences of the Longwall Method of Coal-Getting , 1951 .

[18]  Laurie A. Williams,et al.  Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities , 2011, IEEE Transactions on Software Engineering.

[19]  Laurie A. Williams,et al.  Improving developer activity metrics with issue tracking annotations , 2010, WETSoM '10.

[20]  T. G. Cummings.  Self-Regulating Work Groups: A Socio-Technical Synthesis , 1978 .

[21]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[22]  Premkumar T. Devanbu,et al.  Latent social structure in open source projects , 2008, SIGSOFT '08/FSE-16.

[23]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[24]  Michael Gertz,et al.  Mining email social networks , 2006, MSR '06.

[25]  Andrew Begel,et al.  Codebook: discovering and exploiting relationships in software repositories , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.