Beyond Replication: An example of the potential benefits of replicability in the Mining of Software Repositories Community

While in theory the mining software repositories is an area where replication is easier to perform than for other empirical software engineering fields, a review of papers presented at the Mining Software Repositories Workshop/Working Conference shows that the research studies presented do not satisfy the requirements for easy replication. In this paper, we present some possibilities that replicability may provide to this community that go beyond the verification of results presented in the original study.

[1]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[2]  Tim Menzies,et al.  The \{PROMISE\} Repository of Software Engineering Databases. , 2005 .

[3]  Daniel Izquierdo-Cortazar,et al.  FLOSSMetrics: Free/Libre/Open Source Software Metrics , 2009, 2009 13th European Conference on Software Maintenance and Reengineering.

[4]  Gregorio Robles,et al.  Replicating MSR: A study of the potential replicability of papers published in the Mining Software Repositories proceedings , 2010, 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010).

[5]  Gregorio Robles-Martínez,et al.  Empirical Software Engineering Research on Libre Software: Data Sources, Methodologies and Results , 2012 .

[6]  Kevin Crowston,et al.  Collaboration using OSSmole: a repository of FLOSS data and analyses , 2005, MSR '05.

[7]  Natalia Juristo Juzgado,et al.  Analysis of the influence of communication between researchers on experiment replication , 2006, ISESE '06.

[8]  Christoph Treude,et al.  A comparative exploration of FreeBSD bug lifetimes , 2010, 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010).

[9]  Audris Mockus Amassing and indexing a large sample of version control systems: Towards the census of public source code history , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[10]  Wolfgang John,et al.  Analysis of internet backbone traffic and header anomalies observed , 2007, IMC '07.