Systematicity in Organizational Research Literature Reviews: A Framework and Assessment

In this study, we first develop a framework that presents systematicity as an encompassing orientation toward the application of explicit methods in the practice of literature reviews, informed by the principles of transparency, coverage, saturation, connectedness, universalism, and coherence. We then supplement that conceptual development with empirical insights into the reported practices of systematicity in a sample of 165 published reviews across three journals in organizational research. We finally trace implications for the future conduct of literature reviews, including the potential perils of systematicity without mindfulness.

[1]  E. Tufte The visual display of quantitative information , 1984, The SAGE Encyclopedia of Research Design.

[2]  Torsten Oliver Salge,et al.  Computational Literature Reviews: Method, Algorithms, and Roadmap , 2021, Organizational Research Methods.

[3]  Martin R. W. Hiebl,et al.  Sample Selection in Systematic Literature Reviews of Management Research , 2021, Organizational Research Methods.

[4]  R. Dalal,et al.  Within-Person Variability in Job Performance , 2014, The Handbook of Personality Dynamics and Processes.

[5]  Marc H. Anderson,et al.  Citation Context Analysis as a Method for Conducting Rigorous and Impactful Literature Reviews , 2020, Organizational Research Methods.

[6]  M. Cronin,et al.  The Why and How of the Integrative Review , 2020, Organizational Research Methods.

[7]  M. Alvesson,et al.  Meanings of Theory: Clarifying Theory through Typification , 2020, Journal of Management Studies.

[8]  Martha S. Feldman,et al.  Moving Beyond Templates: A Bricolage Approach to Conducting Trustworthy Qualitative Research , 2020, Organizational Research Methods.

[9]  M. Alvesson,et al.  The Problematizing Review: A Counterpoint to Elsbach and Van Knippenberg's Argument for Integrative Reviews , 2020, Journal of Management Studies.

[10]  Corinne Post,et al.  Advancing Theory with Review Articles , 2020 .

[11]  S. Oliver,et al.  Innovations in framework synthesis as a systematic review method , 2020, Research synthesis methods.

[12]  Andrew Booth,et al.  Qualitative Evidence Synthesis , 2017, SAGE Research Methods Foundations.

[13]  A. Polonioli In search of better science: on the epistemic costs of systematic reviews and the need for a pluralistic stance to literature search , 2019, Scientometrics.

[14]  David W. Bates,et al.  Visualizing Literature Review Theme Evolution on Timeline Maps: Comparison Across Disciplines , 2019, IEEE Access.

[15]  L. Hedges,et al.  How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. , 2019, Annual review of psychology.

[16]  Oliver Baumann,et al.  Effective Search in Rugged Performance Landscapes: A Review and Outlook , 2018, Journal of Management.

[17]  J. McKenzie,et al.  Toward a comprehensive evidence map of overview of systematic review methods: paper 2—risk of bias assessment; synthesis, presentation and summary of the findings; and assessment of the certainty of the evidence , 2018, Systematic Reviews.

[18]  Marc Gruber,et al.  From the Editors—A Brief Primer on Data Visualization Opportunities in Management Research , 2018, Academy of Management Journal.

[19]  Dorothy E. Leidner,et al.  Review and Theory Symbiosis: An Introspective Retrospective , 2018, J. Assoc. Inf. Syst..

[20]  Laura B. Cardinal,et al.  Feature Topic at Organizational Research Methods , 2018 .

[21]  Trisha Greenhalgh,et al.  Time to challenge the spurious hierarchy of systematic over narrative reviews? , 2018, European journal of clinical investigation.

[22]  A. Madden,et al.  Using narrative evidence synthesis in HRM research: An overview of the method, its application, and the lessons learned , 2018 .

[23]  Kimberly D. Elsbach,et al.  Design Thinking and Organizational Culture: A Review and Framework for Future Research , 2018 .

[24]  J. R. Turner,et al.  Theoretical Literature Review: Tracing the Life Cycle of a Theory and Its Verified and Falsified Statements , 2018 .

[25]  Stefan T. Mol,et al.  Text Mining in Organizational Research , 2017, Organizational research methods.

[26]  Stephen H. Bell,et al.  A ?scoping review. , 2018, Sexual health.

[27]  Yin Kia Chiam,et al.  Text-Mining Techniques and Tools for Systematic Literature Reviews: A Systematic Literature Review , 2017, 2017 24th Asia-Pacific Software Engineering Conference (APSEC).

[28]  Ajai S. Gaur,et al.  A systematic approach to conducting review studies: An assessment of content analysis in 25years of IB research , 2017 .

[29]  Harvey A. Whiteford,et al.  How do we define the policy impact of public health research? A systematic review , 2017, Health Research Policy and Systems.

[30]  M. Vaska,et al.  Review Typology: The Basic Types of Reviews for Synthesizing Evidence for the Purpose of Knowledge Translation. , 2017, Journal of the College of Physicians and Surgeons--Pakistan : JCPSP.

[31]  Christian F. Durach,et al.  A New Paradigm for Systematic Literature Reviews in Supply Chain Management , 2017 .

[32]  R. Adams,et al.  Shades of Grey: Guidelines for Working with the Grey Literature in Systematic Reviews for Management and Organizational Studies , 2017 .

[33]  Herman Aguinis,et al.  What You See is What You Get? Enhancing Methodological Transparency in Management Research , 2017 .

[34]  Yu Xiao,et al.  Guidance on Conducting a Systematic Literature Review , 2017 .

[35]  R. Whittemore,et al.  Top 10 Tips for Undertaking Synthesis Research. , 2017, Research in nursing & health.

[36]  Mary Tate,et al.  Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews , 2016, Eur. J. Inf. Syst..

[37]  Monika Kastner,et al.  A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalize the method. , 2016, Journal of clinical epidemiology.

[38]  Neal R Haddaway,et al.  Making literature reviews more reliable through application of lessons from systematic reviews , 2015, Conservation biology : the journal of the Society for Conservation Biology.

[39]  Mike Chiasson,et al.  Avoiding methodological overdose: a declaration for independent ends , 2015, J. Inf. Technol..

[40]  Guy Paré,et al.  Synthesizing information systems knowledge: A typology of literature reviews , 2015, Inf. Manag..

[41]  F. Al-Shamali,et al.  Author Biographies. , 2015, Journal of social work in disability & rehabilitation.

[42]  Thomas G. Cummings,et al.  Scholarly Impact: A Pluralist Conceptualization , 2014 .

[43]  Ivan Zupic,et al.  Bibliometric Methods in Management and Organization , 2014 .

[44]  Frantz Rowe,et al.  What literature review is not: diversity, boundaries and recommendations , 2014, Eur. J. Inf. Syst..

[45]  Jean M. Bartunek,et al.  Academics and Practitioners Are Alike and Unlike , 2014 .

[46]  Ludo Waltman,et al.  Visualizing Bibliometric Networks , 2014 .

[47]  Andrew Booth,et al.  A qualitative evidence synthesis of employees’ views of workplace smoking reduction or cessation interventions , 2013, BMC Public Health.

[48]  J. Norris,et al.  Research synthesis and meta‐analysis , 2012 .

[49]  Rosane Minghim,et al.  A visual analysis approach to validate the selection review of primary studies in systematic reviews , 2012, Inf. Softw. Technol..

[50]  A. Tricco,et al.  What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review , 2012, BMC Medical Research Methodology.

[51]  David Denyer,et al.  Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool , 2012 .

[52]  Emilia Mendes,et al.  Using Visual Text Mining to Support the Study Selection Activity in Systematic Literature Reviews , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[53]  Noel A. Card Applied Meta-Analysis for Social Science Research , 2011 .

[54]  Andrew Booth,et al.  A worked example of "best fit" framework synthesis: A systematic review of views concerning the taking of some potential chemopreventive agents , 2011, BMC medical research methodology.

[55]  Kevin G. Corley,et al.  Building theory about theory building: What constitutes a theoretical contribution? Academy of Management Review, , . , 2011 .

[56]  Andrew Booth,et al.  How much searching is enough? Comprehensive versus optimal retrieval for technology assessments , 2010, International Journal of Technology Assessment in Health Care.

[57]  Donald E. Gibson,et al.  Anger in Organizations: Review and Integration , 2010 .

[58]  Russell Cropanzano,et al.  Writing Nonempirical Articles for Journal of Management: General Thoughts and Suggestions , 2009 .

[59]  Denise M. Rousseau,et al.  Evidence-Based Management: Concept Cleanup Time? , 2009 .

[60]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. , 2009, Journal of clinical epidemiology.

[61]  A. Hillman,et al.  Resource Dependence Theory: A Review , 2009 .

[62]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[63]  Maria J Grant,et al.  A typology of reviews: an analysis of 14 review types and associated methodologies. , 2009, Health information and libraries journal.

[64]  David Clarke,et al.  Advancements in Research Synthesis Methods: From a Methodologically Inclusive Perspective , 2009 .

[65]  K Ann McKibbon,et al.  The capture-mark-recapture technique can be used as a stopping rule when searching in systematic reviews. , 2009, Journal of clinical epidemiology.

[66]  D. Tranfield,et al.  Producing a systematic review. , 2009 .

[67]  P. Hoyningen-Huene Systematicity: The Nature of Science , 2013 .

[68]  Reginald A. Litz,et al.  Stakeholder Theory: Reviewing a Theory That Moves Us , 2008 .

[69]  Jeremy C. Short,et al.  The Application of DICTION to Content Analysis Research in Strategic Management , 2008 .

[70]  D. Rousseau,et al.  Evidence in Management and Organizational Science: Assembling the Field's Full Weight of Scientific Knowledge through Syntheses , 2008 .

[71]  J. Crawford,et al.  The impact of review articles , 2007, Laboratory Investigation.

[72]  A. Edmondson,et al.  METHODOLOGICAL FIT IN MANAGEMENT FIELD RESEARCH. , 2007 .

[73]  T. Judge,et al.  What Causes a Management Article to be Cited—Article, Author, or Journal? , 2007 .

[74]  K. Bowman,et al.  A Research Synthesis Overview , 2007, Nursing science quarterly.

[75]  Rhonda K. Reger,et al.  A Content Analysis of the Content Analysis Literature in Organization Studies: Research Themes, Data Sources, and Methodological Refinements , 2007 .

[76]  A. Pullin,et al.  Guidelines for Systematic Review in Conservation and Environmental Management , 2006, Conservation biology : the journal of the Society for Conservation Biology.

[77]  Thomas J Katz,et al.  Propagation of Errors in Review Articles , 2006, Science.

[78]  Donald D. Bergh,et al.  Some predictors of SMJ article impact , 2006 .

[79]  Trisha Greenhalgh,et al.  Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. , 2005, Social science & medicine.

[80]  H. Arksey,et al.  Scoping studies: towards a methodological framework , 2005 .

[81]  M. MacLure,et al.  ‘Clarity bordering on stupidity’: where’s the quality in systematic review? , 2005 .

[82]  D. Tranfield,et al.  Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review , 2003 .

[83]  Margarete Sandelowski,et al.  Classifying the Findings in Qualitative Studies , 2003, Qualitative health research.

[84]  R. Baumeister,et al.  Writing Narrative Literature Reviews , 1997, Review of General Psychology.

[85]  Samuel B. Bacharach,et al.  Organizational Theories: Some Criteria for Evaluation , 1989 .

[86]  R. Merton,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1973 .

[87]  R. Merton The Normative Structure of Science , 1973 .

[88]  C. Brodsky The Discovery of Grounded Theory: Strategies for Qualitative Research , 1968 .