Tools to support systematic reviews in software engineering: a feature analysis

Background The labour intensive and error prone nature of the systematic review process has led to the development and use of a range of tools to provide automated support. Aim The aim of this research is to evaluate a set of candidate tools that provide support for the overall systematic review process. Method A feature analysis is performed to compare and evaluate four candidate tools. Results Each of the candidates has some strengths and some weaknesses. SLuRp has the highest overall score and SLRTOOL has the lowest overall score. SLuRp scores well on process management features such as support for multiple users and document management and less well on ease of installation. Conclusions Although the tools do not yet support the whole systematic review process they provide a good basis for further development. We suggest a community effort to establish a set of features that can inform future tool development.

[1]  Sandra Camargo Pinto Ferraz Fabbri,et al.  Using GQM and TAM to evaluate StArt - a tool that supports Systematic Review , 2012, CLEI Electron. J..

[2]  Tracy Hall,et al.  SLuRp: a tool to help large complex systematic literature reviews deliver valid and rigorous results , 2012, EAST '12.

[3]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[4]  Tore Dybå,et al.  Applying Systematic Reviews to Diverse Study Types: An Experience Report , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[5]  Pearl Brereton,et al.  Tools to Support Systematic Literature Reviews in Software Engineering: A Mapping Study , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[6]  Mark Staples,et al.  Experiences using systematic review guidelines , 2006, J. Syst. Softw..

[7]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[8]  Jeffrey C. Carver,et al.  Identifying Barriers to the Systematic Literature Review Process , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[9]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[10]  Barbara Ann Kitchenham.  Evaluating software engineering methods and tools, part 7: planning feature analysis evaluation , 1997, SOEN.

[11]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[12]  Pearl Brereton,et al.  Using mapping studies as the basis for further research - A participant-observer case study , 2011, Inf. Softw. Technol..

[13]  Muhammad Ali Babar,et al.  Systematic literature reviews in software engineering: Preliminary results from interviews with researchers , 2009, ESEM 2009.

[14]  Francisca Losavio,et al.  Feature analysis for architectural evaluation methods , 2006, J. Syst. Softw..

[15]  Barbara Kitchenham,et al.  DESMET: a methodology for evaluating software engineering methods and tools , 1997 .

[16]  Cheng Zhang,et al.  Search Engine Overlaps : Do they agree or disagree? , 2007, Second International Workshop on Realising Evidence-Based Software Engineering (REBSE '07).

[17]  Natalia Juristo Juzgado,et al.  Developing search strategies for detecting relevant experiments , 2009, Empirical Software Engineering.

[18]  Barbara Ann Kitchenham,et al.  Evaluating SW Eng. methods and tools, part 8: analysing a feature analysis evaluation , 1997, SOEN.

[19]  Henrik Hedberg,et al.  A preliminary evaluation of software inspection tools, with the DESMET method , 2005, Fifth International Conference on Quality Software (QSIC'05).

[20]  Francisco P. Romero,et al.  SLR-Tool: A Tool for Performing Systematic Literature Reviews , 2010, JISBD.

[21]  Mehwish Riaz,et al.  Experiences Conducting Systematic Reviews from Novices' Perspective , 2010, EASE.