Are systematic reviews better, less biased and of higher quality?

Literature reviews are central to any academic research. Whether journal article, conference presentation or research thesis it is necessary for all of them to provide an overview of earlier research in order to contextualize one’s own findings. More fundamentally, the beginning of any research is crucially dependent on an appropriate literature review. In recent years a supposedly new form of literature review has emerged, so called, systematic reviews. The aim of this paper is to question the key premises of systematic reviews and demonstrate that the claims they are less biased and more rigorous than so called narrative reviews do not hold. This paper briefly introduces the origin of systematic reviews and explains how they are undertaken. Based on this introduction the paper shows that key premises of systematic reviews cannot be fulfilled and that they by no means guaranty the creation of 'better' literature reviews. In contrast, to systematic reviews which put importance on the literature identification and selection process, it argues that reading is central to reviewing literature. Reading enables academics to improve their understanding of the subject area and therefore to further advance their searches. Better literature reviews can only be achieved through better understanding of the subject area. Proper understanding of search techniques will then allow researchers to identify further relevant literature. Reviewing literature is therefore better described as a hermeneutic process.

[1]  M. MacLure,et al.  ‘Clarity bordering on stupidity’: where’s the quality in systematic review? , 2005 .

[2]  T. Greenhalgh,et al.  Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources , 2005, BMJ : British Medical Journal.

[3]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[4]  L. Stewart,et al.  Systematic Reviews: Obtaining data from randomised controlled trials: how much do we need for reliable and informative meta-analyses? , 1994 .

[5]  Robert Fugmann Informationstheorie: Der Jahrhundertbluff Eine zeitkritische Betrachtung (Teil 1) , 2007 .

[6]  Sebastian K. Boell,et al.  Literature Reviews and the Hermeneutic Circle , 2010 .

[7]  邱扬 Hermeneutics , 2022, Hinduism and Tribal Religions.

[8]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[9]  Yogesh Kumar Dwivedi,et al.  Contemporary trends and issues in IT adoption and diffusion research , 2009, J. Inf. Technol..

[10]  James Hartley,et al.  Common weaknesses in traditional abstracts in the social sciences , 2009, J. Assoc. Inf. Sci. Technol..

[11]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[12]  Richard T. Watson,et al.  Introducing MISQ Review - A New Department in MIS Quarterly , 2001, MIS Q..

[13]  Concepción S. Wilson Using online databases to form subject collections for informetric analyses , 1999, Scientometrics.

[14]  Fredric C. Gey,et al.  The Relationship between Recall and Precision , 1994, J. Am. Soc. Inf. Sci..

[15]  P. Herbison,et al.  Problems with meta-analysis. , 1999, The New Zealand medical journal.

[16]  G. F. Mees,et al.  A systematic review of the Indo-Australian Zosteropidae , 1957 .

[17]  Joseph Dunne,et al.  Back to the rough ground , 1992 .

[18]  Concepción S. Wilson,et al.  The scatter of documents over databases in different subject domains: How many databases are needed? , 2001, J. Assoc. Inf. Sci. Technol..

[19]  David Bodoff Emergence of terminological conventions as a searcher-indexer coordination game , 2009, J. Assoc. Inf. Sci. Technol..

[20]  David C. Blair,et al.  Wittgenstein, Language and Information: "Back to the Rough Ground!" , 2006, CoLIS.

[21]  S. Bradford "Sources of information on specific subjects" by S.C. Bradford , 1985 .

[22]  Trisha Greenhalgh,et al.  How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses) , 1997 .

[23]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences , 2006 .

[24]  Stephen J. Bensman Garfield and the impact factor , 2007, Annu. Rev. Inf. Sci. Technol..

[25]  Helen Sharp,et al.  Protocol for a Systematic Literature Review of Motivation in Software Engineering , 2006 .

[26]  Richard N. Shiffman,et al.  Review: Computer-based Guideline Implementation Systems: A Systematic Review of Functionality and Effectiveness , 1999, J. Am. Medical Informatics Assoc..

[27]  G H Guyatt,et al.  Guidelines for reading literature reviews. , 1988, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[28]  Richard J. Boland,et al.  Hermeneutical exegesis in information systems design and use , 2010, Inf. Organ..

[29]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[30]  David C. Blair,et al.  Wittgenstein, Language and Information , 2006 .

[31]  Mark Staples,et al.  Experiences using systematic review guidelines , 2006, J. Syst. Softw..

[32]  Tore Dybå,et al.  Strength of evidence in systematic reviews in software engineering , 2008, ESEM '08.