Dissemination and implementation science in program evaluation: A telemental health clinical consultation case example.

Increased attention has been placed on evaluating the extent to which clinical programs that support the behavioral health needs of youth have effective processes and result in improved patient outcomes. Several theoretical frameworks from dissemination and implementation (D&I) science have been put forth to guide the evaluation of behavioral health program implemented in the context of real-world settings. Although a strong rationale for the integration of D&I science in program evaluation exists, few examples exist available to guide the evaluator in integrating D&I science in the planning and execution of evaluation activities. This paper seeks to inform program evaluation efforts by outlining two D&I frameworks and describing their integration in program evaluation design. Specifically, this paper seeks to support evaluation efforts by illustrating the use of these frameworks via a case example of a telemental health consultation program in pediatric primary care designed to improve access to behavioral health care for children and adolescents in rural settings. Lessons learned from this effort, as well as recommendations regarding the future evaluation of programs using D&I science to support behavioral health care in community-based settings are discussed.

[1]  R. Berenson,et al.  Achieving the Potential of Health Care Performance Measures , 2013 .

[2]  E. Nelson,et al.  Using Videoconferencing to Provide Psychological Services to Rural Children and Adolescents: A Review and Case Example , 2014, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[3]  M. R. Lopez-Patton,et al.  Translating evidence-based interventions from research to practice: challenges and lessons learned , 2015, Translational behavioral medicine.

[4]  Alan E Kazdin,et al.  Evidence-based treatment and practice: new opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. , 2008, The American psychologist.

[5]  J. Westfall,et al.  Practice-based research--"Blue Highways" on the NIH roadmap. , 2007, JAMA.

[6]  T. Young,et al.  Effectiveness of school-based telehealth care in urban and rural elementary schools. , 2003, Pediatrics.

[7]  S. Gabel The integration of mental health into pediatric practice: pediatricians and child and adolescent psychiatrists working together in new models of care. , 2010, The Journal of pediatrics.

[8]  J. Kirchner,et al.  A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project , 2015, Implementation Science.

[9]  W. Gardner,et al.  The Physician Belief Scale and psychosocial problems in children: a report from the Pediatric Research in Office Settings and the Ambulatory Sentinel Practice Network. , 1999, Journal of developmental and behavioral pediatrics : JDBP.

[10]  Michael Scriven,et al.  Evaluation Models: Viewpoints on Educational and Human Services Evaluation , 2000 .

[11]  D. Drotar,et al.  Steps toward a clinically relevant science of interventions in pediatric settings: introduction to the special issue. , 2001, Journal of pediatric psychology.

[12]  R. Glasgow,et al.  The RE-AIM framework: a systematic review of use over time. , 2013, American journal of public health.

[13]  S. Schoenwald,et al.  Effectiveness, transportability, and dissemination of interventions: what matters when? , 2001, Psychiatric services.

[14]  D. Bird ADDRESSING MENTAL HEALTH WORKFORCE NEEDS IN UNDERSERVED RURAL AREAS: ACCOMPLISHMENTS AND CHALLENGES , 2002 .

[15]  David A Chambers,et al.  Bridging research and practice: models for dissemination and implementation research. , 2012, American journal of preventive medicine.

[16]  Jeffrey R. Harris,et al.  A Framework for Disseminating Evidence-Based Health Promotion Practices , 2011, Preventing chronic disease.

[17]  Jon D. Elhai,et al.  Attitudes Toward Medical and Mental Health Care Delivered Via Telehealth Applications Among Rural and Urban Primary Care Patients , 2008, The Journal of nervous and mental disease.

[18]  D. Janicke,et al.  Treatment outcome research in rural pediatric populations: the challenge of recruitment. , 2011, Journal of pediatric psychology.

[19]  William T Riley,et al.  Pragmatic measures: what they are and why we need them. , 2013, American journal of preventive medicine.

[20]  Michael Scriven Prose and Cons about Goal-Free Evaluation: , 1991 .

[21]  Russell E. Glasgow,et al.  Work Site Health Promotion Research: To what Extent can we Generalize the Results and what is Needed to Translate Research to Practice? , 2003, Health education & behavior : the official publication of the Society for Public Health Education.

[22]  Emil J. Posavac,et al.  Program Evaluation: Methods and Case Studies , 1980 .

[23]  Allied Professions Atlas : child and adolescent mental health resources : global concerns : implications for the future , 2005 .

[24]  D. Satcher Mental Health: A Report of the Surgen General , 1999 .

[25]  C. Johnston,et al.  Family physicians' involvement and self-reported comfort and skill in care of children with behavioral and emotional problems: a population-based survey , 2005, BMC family practice.

[26]  K. McDonald Considering context in quality improvement interventions and implementation: concepts, frameworks, and application. , 2013, Academic pediatrics.

[27]  D. Drotar,et al.  Conducting psychological intervention research in pediatric clinical settings: Strategies and implications. , 2014 .

[28]  David A. Chambers,et al.  Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges , 2008, Administration and Policy in Mental Health and Mental Health Services Research.

[29]  E. Suchman Evaluative Research: Principles and Practice in Public Service and Social Action Progr , 1968 .

[30]  W. Gardner,et al.  Management of pediatric mental disorders in primary care: where are we now and where are we going? , 2006, Current opinion in pediatrics.

[31]  Thomas R Kratochwill,et al.  Preparing psychologists for evidence-based school practice: lessons learned and challenges ahead. , 2007, The American psychologist.

[32]  D. Simpson,et al.  Assessing organizational readiness for change. , 2002, Journal of substance abuse treatment.

[33]  Ross C Brownson,et al.  A glossary for dissemination and implementation research in health. , 2008, Journal of public health management and practice : JPHMP.

[34]  G. Robert,et al.  Diffusion of innovations in service organizations: systematic review and recommendations. , 2004, The Milbank quarterly.

[35]  Dean L. Fixsen,et al.  Core Implementation Components , 2009 .

[36]  Fiona Gillison,et al.  Mixed method evaluation of a community-based physical activity program using the RE-AIM framework: Practical application in a real-world setting , 2015, BMC Public Health.

[37]  Debbe Thompson,et al.  A framework for evaluating eHealth research. , 2006, Evaluation and program planning.

[38]  P. Yellowlees,et al.  The child and adolescent telepsychiatry consultation: can it be a more effective clinical process for certain patients than conventional practice? , 2010, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[39]  R. Glasgow,et al.  Evaluating the public health impact of health promotion interventions: the RE-AIM framework. , 1999, American journal of public health.

[40]  Ruth P Saunders,et al.  Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide , 2005, Health promotion practice.

[41]  K. Hoagwood,et al.  Training and Education in Clinical Psychology in the Context of the Patient Protection and Affordable Care Act , 2014 .

[42]  W. E. Craighead,et al.  Establishing a Telepsychiatry Consultation Practice in Rural Georgia for Primary Care Physicians , 2012, Clinical pediatrics.

[43]  David D. Royse,et al.  Program Evaluation: An Introduction , 1992 .

[44]  T. Eng,et al.  eHealth Research and Evaluation: Challenges and Opportunities , 2002, Journal of health communication.

[45]  K. Myers,et al.  Child and adolescent telepsychiatry: variations in utilization, referral patterns and practice trends , 2010, Journal of telemedicine and telecare.

[46]  K. Myers,et al.  Feasibility, acceptability, and sustainability of telepsychiatry for children and adolescents. , 2007, Psychiatric services.

[47]  Bryan J Weiner,et al.  Instrumentation issues in implementation science , 2014, Implementation Science.

[48]  Lisa M Klesges,et al.  What Does It Mean to “Employ” the RE-AIM Model? , 2013, Evaluation & the health professions.

[49]  E. Proctor Keynote Address: Social Work Research and the Quest for Effective Practice , 1998 .

[50]  T. Kosten,et al.  Developing the tools of implementation science in substance use disorders treatment: applications of the consolidated framework for implementation research. , 2011, Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors.

[51]  G. Aarons,et al.  Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[52]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[53]  R. Poropatich,et al.  A standard telemental health evaluation model: the time is now. , 2012, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[54]  J. Norcross,et al.  What clinical psychologists know about evidence-based practice: familiarity with online resources and research methods. , 2011, Journal of clinical psychology.

[55]  Lawrence A. Palinkas,et al.  Mixed Method Designs in Implementation Research , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[56]  Karen A. Blase,et al.  Implementation Research: A Synthesis of the , 2005 .

[57]  D. Montaño,et al.  A scale to measure physician beliefs about psychosocial aspects of patient care. , 1984, Social science & medicine.