Systematic Mapping Studies in Software Engineering

BACKGROUND: A software engineering systematic map is a defined method to build a classification scheme and structure a software engineering field of interest. The analysis of results focuses on frequencies of publications for categories within the scheme. Thereby, the coverage of the research field can be determined. Different facets of the scheme can also be combined to answer more specific research questions. OBJECTIVE: We describe how to conduct a systematic mapping study in software engineering and provide guidelines. We also compare systematic maps and systematic reviews to clarify how to chose between them. This comparison leads to a set of guidelines for systematic maps. METHOD: We have defined a systematic mapping process and applied it to complete a systematic mapping study. Furthermore, we compare systematic maps with systematic reviews by systematically analyzing existing systematic reviews. RESULTS: We describe a process for software engineering systematic mapping studies and compare it to systematic reviews. Based on this, guidelines for conducting systematic maps are defined. CONCLUSIONS: Systematic maps and reviews are different in terms of goals, breadth, validity issues and implications. Thus, they should be used complementarily and require different methods (e.g., for analysis).

[1]  R. Feldt,et al.  Software Product Line Variability: A Systematic Mapping Study , 2008 .

[2]  Natalia Juristo Juzgado,et al.  Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[3]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[4]  Stephen G. MacDonell,et al.  Comparing Local and Global Software Effort Estimation Models -- Reflections on a Systematic Review , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[5]  Roel Wieringa,et al.  Requirements engineering paper classification and evaluation criteria: a proposal and a discussion , 2005, Requirements Engineering.

[6]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[7]  Guilherme Horta Travassos,et al.  Cross versus Within-Company Cost Estimation Studies: A Systematic Review , 2007, IEEE Transactions on Software Engineering.

[8]  Pearl Brereton,et al.  Evaluation and assessment in software engineering , 1997, J. Syst. Softw..

[9]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[10]  Pearl Brereton,et al.  Evidence relating to Object-Oriented software design: A survey , 2007, ESEM 2007.

[11]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[12]  David R. Jones,et al.  Synthesising qualitative and quantitative evidence: A review of possible methods , 2005 .

[13]  Magne Jørgensen,et al.  Software effort estimation terminology: The tower of Babel , 2006, Inf. Softw. Technol..

[14]  Stephen G. MacDonell,et al.  Comparing Local and Global Software Effort Estimation Models -- Reflections on a Systematic Review , 2007, ESEM 2007.

[15]  Tore Dybå,et al.  A Systematic Review of Theory Use in Software Engineering Experiments , 2007, IEEE Transactions on Software Engineering.

[16]  Emilia Mendes,et al.  A systematic review of Web engineering research , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[17]  Venkataraman Ramesh,et al.  Research in software engineering: an analysis of the literature , 2002, Inf. Softw. Technol..

[18]  Pearl Brereton,et al.  Evidence relating to Object-Oriented software design: A survey , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[19]  Tore Dybå,et al.  A systematic review of effect size in software engineering experiments , 2007, Inf. Softw. Technol..

[20]  Pearl Brereton,et al.  Preliminary results of a study of the completeness and clarity of structured abstracts , 2007, EASE.

[21]  Tore Dybå,et al.  A systematic review of statistical power in software engineering experiments , 2006, Inf. Softw. Technol..