The difficulties of systematic reviews

The need for robust evidence to support conservation actions has driven the adoption of systematic approaches to research synthesis in ecology. However, applying systematic review to complex or open questions remains challenging, and this task is becoming more difficult as the quantity of scientific literature increases. We drew on the science of linguistics for guidance as to why the process of identifying and sorting information during systematic review remains so labor intensive, and to provide potential solutions. Several linguistic properties of peer-reviewed corpora-including nonrandom selection of review topics, small-world properties of semantic networks, and spatiotemporal variation in word meaning-greatly increase the effort needed to complete the systematic review process. Conversely, the resolution of these semantic complexities is a common motivation for narrative reviews, but this process is rarely enacted with the rigor applied during linguistic analysis. Therefore, linguistics provides a unifying framework for understanding some key challenges of systematic review and highlights 2 useful directions for future research. First, in cases where semantic complexity generates barriers to synthesis, ecologists should consider drawing on existing methods-such as natural language processing or the construction of research thesauri and ontologies-that provide tools for mapping and resolving that complexity. These tools could help individual researchers classify research material in a more robust manner and provide valuable guidance for future researchers on that topic. Second, a linguistic perspective highlights that scientific writing is a rich resource worthy of detailed study, an observation that can sometimes be lost during the search for data during systematic review or meta-analysis. For example, mapping semantic networks can reveal redundancy and complementarity among scientific concepts, leading to new insights and research questions. Consequently, wider adoption of linguistic approaches may facilitate improved rigor and richness in research synthesis.

[1]  D. Driscoll,et al.  A succession of theories: purging redundancy from disturbance theory , 2016, Biological reviews of the Cambridge Philosophical Society.

[2]  Christopher J. Lortie,et al.  Formalized synthesis opportunities for ecology: systematic reviews and meta‐analyses , 2014 .

[3]  Andrew J Tyre,et al.  Evaluating the efficacy of adaptive management approaches: is there a formula for success? , 2011, Journal of environmental management.

[4]  A. Pullin,et al.  Doing more good than harm: building an evidence-base for conservation and environmental management. , 2009 .

[5]  P. Sherman,et al.  Ecological and evolutionary traps. , 2002 .

[6]  R Mac Nally,et al.  Vive la résistance: reviving resistance for 21st century conservation. , 2015, Trends in ecology & evolution.

[7]  Joshua B. Tenenbaum,et al.  The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth , 2001, Cogn. Sci..

[8]  Harry Eugene Stanley,et al.  Languages cool as they expand: Allometric scaling and the decreasing need for new words , 2012, Scientific Reports.

[9]  David B. Lindenmayer,et al.  Integrating plant- and animal-based perspectives for more effective restoration of biodiversity , 2016 .

[10]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[11]  David B Lindenmayer,et al.  Adaptive monitoring: a new paradigm for long-term research and monitoring. , 2009, Trends in ecology & evolution.

[12]  C. Bradshaw,et al.  Density dependence: an ecological Tower of Babel , 2012, Oecologia.

[13]  D. Lindenmayer,et al.  The ecology, distribution, conservation and management of large old trees , 2017, Biological reviews of the Cambridge Philosophical Society.

[14]  David B. Lindenmayer,et al.  Framework to improve the application of theory in ecology and conservation , 2012 .

[15]  Jason M Tylianakis,et al.  Specialization and Rarity Predict Nonrandom Loss of Interactions from Mutualist Networks , 2012, Science.

[16]  S. Danziger,et al.  Extraneous factors in judicial decisions , 2011, Proceedings of the National Academy of Sciences.

[17]  W. Sutherland,et al.  The need for evidence-based conservation. , 2004, Trends in ecology & evolution.

[18]  Neal R Haddaway,et al.  Making literature reviews more reliable through application of lessons from systematic reviews , 2015, Conservation biology : the journal of the Society for Conservation Biology.

[19]  William J Sutherland,et al.  Organising evidence for environmental management decisions: a '4S' hierarchy. , 2014, Trends in ecology & evolution.

[20]  Martin J. Westgate,et al.  Text analysis tools for identification of emerging topics and research gaps in conservation science , 2015, Conservation biology : the journal of the Society for Conservation Biology.

[21]  Guillaume Bastille-Rousseau,et al.  The evolution of peer review as a basis for scientific publication: directional selection towards a robust discipline? , 2016, Biological reviews of the Cambridge Philosophical Society.

[22]  J C Ogden,et al.  Ecological conceptual models: a framework and case study on ecosystem management for South Florida sustainability. , 2001, The Science of the total environment.

[23]  Eric Garnier,et al.  PLANT FUNCTIONAL MARKERS CAPTURE ECOSYSTEM PROPERTIES DURING SECONDARY SUCCESSION , 2004 .

[24]  Veronica A. J. Doerr,et al.  Maximizing the value of systematic reviews in ecology when data or resources are limited , 2015 .

[25]  M. Baine,et al.  Artificial reefs: a review of their design, application, management and performance , 2001 .

[26]  Bryan C. Pijanowski,et al.  Automated content analysis: addressing the big literature challenge in ecology and evolution , 2016 .

[27]  Martin J. Westgate,et al.  Optimal taxonomic groups for biodiversity assessment: a meta‐analytic approach , 2017 .

[28]  Neal R Haddaway,et al.  The benefits of systematic mapping to evidence-based environmental management , 2016, Ambio.

[29]  Martin J. Westgate,et al.  Adaptive management of biological systems: A review , 2013 .

[30]  John Watkins,et al.  Automated tagging of environmental data using a novel SKOS formatted environmental thesaurus , 2014, Earth Science Informatics.

[31]  Neal R Haddaway,et al.  The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching , 2015, PloS one.

[32]  Dominique Gravel,et al.  Beyond species: why ecological interaction networks vary through space and time , 2014, bioRxiv.

[33]  A. Pullin,et al.  Guidelines for Systematic Review in Conservation and Environmental Management , 2006, Conservation biology : the journal of the Society for Conservation Biology.