Guidelines for snowballing in systematic literature studies and a replication in software engineering

Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably. Objective: This paper presents guidelines for conducting literature reviews using a snowballing approach, and they are illustrated and evaluated by replicating a published systematic literature review. Method: The guidelines are based on the experience from conducting several systematic literature reviews and experimenting with different approaches. Results: The guidelines for using snowballing as a way to search for relevant literature was successfully applied to a systematic literature review. Conclusions: It is concluded that using snowballing, as a first search strategy, may very well be a good alternative to the use of database searches.

[1]  G. M. Meier.  Leading issues in economic development , 1971 .

[2]  A. Krueger.  Debt, Capital Flows, and LDC Growth , 1987 .

[3]  S. Fischer.  Sharing the Burden of the International Debt Crisis , 1987 .

[4]  J. Anand,et al.  Technological Capabilities of Countries, Firm Rivalry and Foreign Direct Investment , 1997 .

[5]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[6]  D. Rodrik,et al.  Short-Term Capital Flows , 1999 .

[7]  Campbell R. Harvey,et al.  The Asian Bet , 1999 .

[8]  Will Hayes,et al.  Research synthesis in software engineering: a case for meta-analysis , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[9]  James Miller.  Can results from software engineering experiments be safely combined? , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[10]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[11]  Jiawen Yang.  Financial Globalization and its Impact on Emerging Markets , 2000 .

[12]  R. Albuquerque.  The Composition of International Capital Flows: Risk Sharing Through Foreign Direct Investment , 2002 .

[13]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[14]  Michael L. Mussa.  Global Economic Prospects , 2002 .

[15]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[16]  T. Greenhalgh,et al.  Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources , 2005, BMJ : British Medical Journal.

[17]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[18]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[19]  Per Runeson,et al.  Reference-based search strategies in systematic reviews , 2009, EASE.

[20]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[21]  Emilia Mendes,et al.  How Reliable Are Systematic Reviews in Empirical Software Engineering? , 2010, IEEE Transactions on Software Engineering.

[22]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A tertiary study , 2010, Inf. Softw. Technol..

[23]  Pearl Brereton,et al.  Using mapping studies as the basis for further research - A participant-observer case study , 2011, Inf. Softw. Technol..

[24]  Pearl Brereton,et al.  Repeatability of systematic literature reviews , 2011, EASE.

[25]  Muhammad Ali Babar,et al.  Identifying relevant studies in software engineering , 2011, Inf. Softw. Technol..

[26]  Claes Wohlin,et al.  An empirically based terminology and taxonomy for global software engineering , 2014, Empirical Software Engineering.

[27]  Claes Wohlin,et al.  Global software engineering and agile practices: a systematic review , 2012, J. Softw. Evol. Process..

[28]  Claes Wohlin,et al.  Systematic literature studies: Database searches vs. backward snowballing , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[29]  Claes Wohlin,et al.  On the reliability of mapping studies in software engineering , 2013, J. Syst. Softw..

[30]  Claes Wohlin,et al.  Systematic literature reviews in software engineering , 2013, Inf. Softw. Technol..