Analysis of the influence of communication between researchers on experiment replication

The replication of experiments is a key undertaking in SE. Successful replications enable a discipline's body of knowledge to grow, as the results are added to those of earlier replications. However, replication is extremely difficult in SE, primarily because it is difficult to get a setting that is exactly the same as in the original experiment. Consequently, changes have to be made to the experiment to adapt it to the new site. To be able to replicate an experiment, information also has to be transmitted (usually orally and in writing) between the researchers who ran the experiment earlier and the ones who are going to replicate the experiment. This article examines the influence of the type of communication there is between experimenters on how successful a replication is. We have studied three replications of the same experiment in which different types of communication were used.

[1]  Natalia Juristo Juzgado,et al.  Functional Testing, Structural Testing, and Code Reading: What Fault Type Do They Each Detect? , 2003, ESERNET.

[2]  Erik Kamsties,et al.  An Empirical Evaluation of Three Defect-Detection Techniques , 1995, ESEC.

[3]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[4]  Ana M. Moreno,et al.  Lecture Notes on Empirical Software Engineering , 2003, Series on Software Engineering and Knowledge Engineering.

[5]  James Miller,et al.  Comparing and combining software defect detection techniques: a replicated empirical study , 1997, ESEC '97/FSE-5.

[6]  James Miller,et al.  Applying meta-analytical procedures to software engineering experiments , 2000, J. Syst. Softw..

[7]  A. R. Ilersic,et al.  Research methods in social relations , 1961 .

[8]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[9]  Jeffrey C. Carver,et al.  A Pragmatic Documents Standard for an Experience Library: Roles,Documen, Contents and Structure , 2001 .

[10]  Oliver Laitenberger,et al.  (Quasi-)Experimental Studies in Industrial Settings , 2003, Lecture Notes on Empirical Software Engineering.

[11]  Victor R. Basili,et al.  Comparing the Effectiveness of Software Testing Strategies , 1987, IEEE Transactions on Software Engineering.

[12]  Jeffrey C. Carver,et al.  Knowledge-Sharing Issues in Experimental Software Engineering , 2004, Empirical Software Engineering.

[13]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[14]  Adam A. Porter,et al.  Assessing Software Review Meetings: Results of a Comparative Analysis of Two Experimental Studies , 1997, IEEE Trans. Software Eng..

[15]  James Miller,et al.  An empirical evaluation of defect detection techniques , 1997, Inf. Softw. Technol..

[16]  Natalia Juristo Juzgado,et al.  Reviewing 25 Years of Testing Technique Experiments , 2004, Empirical Software Engineering.

[17]  Eliot R. Smith,et al.  Research methods in social relations , 1962 .

[18]  Magne Jørgensen,et al.  A review of studies on expert estimation of software development effort , 2004, J. Syst. Softw..

[19]  Jeffrey C. Carver,et al.  Replicated Studies: Building a Body of Knowledge about Software Reading Techniques , 2003, Lecture Notes on Empirical Software Engineering.