Approaches to Mixed Methods Dissemination and Implementation Research: Methods, Strengths, Caveats, and Opportunities

Limited translation of research into practice has prompted study of diffusion and implementation, and development of effective methods of encouraging adoption, dissemination and implementation. Mixed methods techniques offer approaches for assessing and addressing processes affecting implementation of evidence-based interventions. We describe common mixed methods approaches used in dissemination and implementation research, discuss strengths and limitations of mixed methods approaches to data collection, and suggest promising methods not yet widely used in implementation research. We review qualitative, quantitative, and hybrid approaches to mixed methods dissemination and implementation studies, and describe methods for integrating multiple methods to increase depth of understanding while improving reliability and validity of findings.

[1]  R. Yin Enhancing the quality of case studies in health services research. , 1999, Health services research.

[2]  I. Bourgeault,et al.  The SAGE Handbook of Qualitative Methods in Health Research , 2010 .

[3]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[4]  Allan Donner,et al.  Cluster randomization trials in epidemiology: theory and application , 1994 .

[5]  P. M. Ginter,et al.  Health departments' implementation of public health's core functions: an assessment of health impacts. , 2005, Public health.

[6]  J. Lellouch,et al.  Explanatory and pragmatic attitudes in therapeutical trials. , 1967, Journal of chronic diseases.

[7]  H Robbins,et al.  Clinical and prophylactic trials with assured new treatment for those at greater risk: II. Examples. , 1996, American journal of public health.

[8]  R. Yin Case Study Research: Design and Methods , 1984 .

[9]  N. Denzin,et al.  Handbook of Qualitative Research , 1994 .

[10]  A. Kimball Romney,et al.  Systematic Data Collection , 1988 .

[11]  Tom W. Smith,et al.  ASKING SENSITIVE QUESTIONS THE IMPACT OF DATA COLLECTION MODE, QUESTION FORMAT, AND QUESTION CONTEXT , 1996 .

[12]  D. Davies,et al.  Qualitative Research and the Question of Rigor , 2002, Qualitative health research.

[13]  W. Neuman,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2002 .

[14]  H. Robbins,et al.  Clinical and prophylactic trials with assured new treatment for those at greater risk: I. A design proposal. , 1996, American journal of public health.

[15]  M. Hotopf The pragmatic randomised controlled trial , 2002 .

[16]  L. Blankertz The Value and Practicality of Deliberate Sampling for Heterogeneity: A Critical Multiplist Perspective , 1998 .

[17]  K. Charmaz,et al.  Constructing Grounded Theory , 2014 .

[18]  Michael J. Kolen,et al.  Comparisons of Methodologies and Results in Vertical Scaling for Educational Achievement Tests , 2007 .

[19]  J. Abelson,et al.  Development of a framework for effective community engagement in Ontario, Canada. , 2011, Health policy.

[20]  M. Angrosino,et al.  Recontextualizing Observation: Ethnography, Pedagogy, and the Prospects for a Progressive Political Agenda. , 2005 .

[21]  J. Gabbay,et al.  Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care , 2004, BMJ : British Medical Journal.

[22]  James A. Holstein,et al.  Handbook of Interview Research , 2001 .

[23]  J. Morse Qualitative data analysis (2nd ed): Mathew B. Miles and A. Michael Huberman. Thousand Oaks, CA: Sage Publications, 1994. Price: $65.00 hardback, $32.00 paperback. 238 pp , 1996 .

[24]  Naihua Duan,et al.  Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research , 2015, Administration and Policy in Mental Health and Mental Health Services Research.

[25]  Claire B Draucker,et al.  Theoretical Sampling and Category Development in Grounded Theory , 2007, Qualitative health research.

[26]  Pennie G. Foster-Fishman,et al.  Methods of Rapid Evaluation, Assessment, and Appraisal , 2007 .

[27]  Alan M. Zaslavsky,et al.  Using Telephone Interviews to Reduce Nonresponse Bias to Mail Surveys of Health Plan Members , 2002, Medical care.

[28]  J. Westfall,et al.  Practice-based research--"Blue Highways" on the NIH roadmap. , 2007, JAMA.

[29]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[30]  Brian K. Gran,et al.  Complexity, Generality, and Qualitative Comparative Analysis , 2003 .

[31]  L. Behar,et al.  Defining Community Readiness for the Implementation of a System of Care , 2009, Administration and Policy in Mental Health and Mental Health Services Research.

[32]  D. McCaffrey,et al.  National probability samples in studies of low-prevalence diseases. Part II: Designing and implementing the HIV cost and services utilization study sample. , 1999, Health services research.

[33]  B. Poland Transcription Quality as an Aspect of Rigor in Qualitative Research , 1995 .

[34]  H. Wiegand,et al.  Kish, L.: Survey Sampling. John Wiley & Sons, Inc., New York, London 1965, IX + 643 S., 31 Abb., 56 Tab., Preis 83 s. , 1968 .

[35]  Peter G. M. van der Heijden,et al.  Meta-Analysis of Randomized Response Research , 2005 .

[36]  N. Denzin,et al.  The Sage handbook of qualitative research, 3rd ed. , 2005 .

[37]  William M. K. Trochim,et al.  An introduction to concept mapping for planning and evaluation. , 1989 .

[38]  M. Marshall,et al.  The key informant technique. , 1996, Family practice.

[39]  Alexa Jane Trumpy,et al.  Subject to Negotiation: The Mechanisms Behind Co-Optation and Corporate Reform , 2008 .

[40]  G. McHugo,et al.  Evidence-Based Practice Implementation in Community Mental Health Settings: The Relative Importance of Key Domains of Implementation Activity , 2012, Administration and Policy in Mental Health and Mental Health Services Research.

[41]  P. V. Marsden,et al.  Handbook of Survey Research , 1985 .

[42]  Allan Donner,et al.  Some aspects of the design and analysis of cluster randomization trials , 2002 .

[43]  D. Morgan Successful Focus Groups: Advancing the State of the Art , 1993 .

[44]  James D. Wright,et al.  Handbook of Survey Research. , 1985 .

[45]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[46]  Candice Bowman,et al.  The role of formative evaluation in implementation research and the QUERI experience , 2006, Journal of General Internal Medicine.

[47]  Jelke Bethlehem,et al.  Computer Assisted Survey Information Collection , 1998 .

[48]  Charles C. Ragin,et al.  Using qualitative comparative analysis to study causal complexity. , 1999, Health services research.

[49]  David R. Holtgrave,et al.  Alternatives to the randomized controlled trial. , 2008, American journal of public health.

[50]  H. Bernard Research Methods in Anthropology: Qualitative and Quantitative Approaches , 1988 .

[51]  B MilesMatthew,et al.  Qualitative Data Analysis , 2009, Approaches and Processes of Social Science Research.

[52]  L. Krefting Rigor in qualitative research: the assessment of trustworthiness. , 1991, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[53]  Leslie Kish,et al.  Survey sampling / Leslie Kish , 1995 .

[54]  D. Weiss Adaptive testing by computer. , 1985, Journal of consulting and clinical psychology.

[55]  William B. Stevenson,et al.  The Formal Analysis of Narratives of Organizational Change , 1998 .

[56]  K. Charmaz,et al.  Qualitative interviewing and grounded theory analysis , 2012 .

[57]  Kathleen M. Quinlan,et al.  Building a local agenda for dissemination of research into practice. , 2008, Journal of public health management and practice : JPHMP.

[58]  R. Whittemore,et al.  Validity in Qualitative Research , 2001, Qualitative health research.

[59]  V. Gilchrist Key informant interviews. , 1992 .

[60]  Susan E. Chase,et al.  Narrative Inquiry: Multiple Lenses, Approaches, Voices. , 2005 .

[61]  D. Rubin,et al.  Identification of Causal Effects Using Instrumental Variables: Rejoinder , 1996 .

[62]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[63]  Mick P. Couper,et al.  Designing Effective Web Surveys: Preface , 2008 .

[64]  David S. Lee,et al.  Regression Discontinuity Designs in Economics , 2009 .

[65]  S. Shiffman,et al.  Ecological momentary assessment. , 2008, Annual review of clinical psychology.

[66]  D. Metzger,et al.  Rapid assessment of the HIV/AIDS crisis in racial and ethnic minority communities: an approach for timely community interventions. , 2003, American journal of public health.

[67]  John W. Creswell,et al.  Designing and Conducting Mixed Methods Research , 2006 .

[68]  Edgar F. Borgatta,et al.  Sociological Methodology, 1969. , 1969 .

[69]  Elizabeth Allen,et al.  Process evaluation in randomised controlled trials of complex interventions , 2006, BMJ : British Medical Journal.

[70]  Carol A. Caronna Why Use Qualitative Methods to Study Health Care Organizations? Insights from Multi-Level Case Studies , 2010 .

[71]  Joshua D. Angrist,et al.  Identification of Causal Effects Using Instrumental Variables , 1993 .

[72]  S M Shortell,et al.  The emergence of qualitative methods in health services research. , 1999, Health services research.

[73]  D. R. Heise Modeling event structures , 1989 .

[74]  D. Stryer,et al.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. , 2003, JAMA.

[75]  B. Crabtree,et al.  The qualitative research interview , 2006, Medical education.

[76]  Merrill Singer,et al.  A Methodological Model for Rapid Assessment, Response, and Evaluation: The RARE Program in Public Health , 2001 .

[77]  S. Murray,et al.  Listening to local voices: adapting rapid appraisal to assess health and social needs in general practice , 1994, BMJ.

[78]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[79]  R. Stake Qualitative Case Studies. , 2005 .

[80]  Bryan R Luce,et al.  Rethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change , 2009, Annals of Internal Medicine.

[81]  Lawrence A Palinkas,et al.  Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. , 2009, American journal of public health.

[82]  M. Tremblay The Key Informant Technique: A Nonethnographic Application , 1957 .

[83]  R. Fisher Statistical methods for research workers , 1927, Protoplasma.

[84]  Ellen Frank,et al.  Development of a computerized adaptive test for depression. , 2012, Archives of general psychiatry.

[85]  Jong Bae Kim,et al.  Computerized adaptive test-depression inventory not ready for prime time-reply. , 2013, JAMA psychiatry.

[86]  Robert K. Yin,et al.  Applications of case study research , 1993 .

[87]  D. Silverman Doing Qualitative Research , 2009 .

[88]  Floyd J. Fowler,et al.  Survey Research Methods , 1984 .

[89]  Greg Dimitriadis,et al.  Focus Groups: Strategic Articulations of Pedagogy, Politics, and Inquiry. , 2005 .

[90]  Shortell Sm,et al.  The emergence of qualitative methods in health services research. , 1999 .

[91]  Robert Korstad,et al.  Historical Inference and Event-Structure Analysis , 1998 .

[92]  G. Willis,et al.  Research Synthesis: The Practice of Cognitive Interviewing , 2007 .

[93]  James Beebe,et al.  Rapid Assessment Process: An Introduction , 2001 .

[94]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[95]  N. Denzin,et al.  Collecting and Interpreting Qualitative Materials , 2012 .

[96]  D. Campbell,et al.  Regression-Discontinuity Analysis: An Alternative to the Ex-Post Facto Experiment , 1960 .

[97]  Charles C. Ragin,et al.  TURNING THE TABLES: HOW CASE-ORIENTED RESEARCH CHALLENGES VARIABLE-ORIENTED RESEARCH , 1997 .

[98]  K. Wade,et al.  Practice-Based Research , 2007, Social work in health care.

[99]  D. Berwick Disseminating innovations in health care. , 2003, JAMA.

[100]  David A. Chambers,et al.  Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges , 2008, Administration and Policy in Mental Health and Mental Health Services Research.

[101]  D. R. Heise,et al.  Event Structure Models from Ethnographic Data , 1990 .

[102]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[103]  J. Creswell Qualitative inquiry and research design: choosing among five traditions. , 1998 .

[104]  Lawrence A. Palinkas,et al.  Mixed Method Designs in Implementation Research , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[105]  David M. Murray,et al.  Design and Analysis of Group- Randomized Trials , 1998 .

[106]  J. Morse Determining Sample Size , 2000 .

[107]  Charles C. Ragin,et al.  The distinctiveness of case-oriented research. , 1999, Health services research.

[109]  G. Aarons,et al.  Adaptation happens: a qualitative case study of implementation of The Incredible Years evidence‐based parent training programme in a residential substance abuse treatment programme , 2012 .

[110]  Lawrence A. Palinkas,et al.  Mixed-methods designs in mental health services research: a review. , 2011, Psychiatric services.

[111]  Earl R. Babbie,et al.  Survey Research Methods , 1984 .

[112]  Ian Harvey,et al.  A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. , 2009, Journal of clinical epidemiology.

[113]  B. Pentland Building Process Theory with Narrative: from Description to Explanation , 1999 .

[114]  Sylwia Męcfal Recenzja książki. Robert K. yin, Case Study Research. Design and Methods (fourth Edition), thousand Oaks, CA: Sage Publications, 2009 , 2012 .

[115]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[116]  J. Morse,et al.  Verification Strategies for Establishing Reliability and Validity in Qualitative Research , 2002 .

[117]  E. Leeuw,et al.  To mix or not to mix data collection modes in surveys. , 2005 .