Development of a framework and coding system for modifications and adaptations of evidence-based interventions

BackgroundEvidence-based interventions are frequently modified or adapted during the implementation process. Changes may be made to protocols to meet the needs of the target population or address differences between the context in which the intervention was originally designed and the one into which it is implemented [Addict Behav 2011, 36(6):630–635]. However, whether modification compromises or enhances the desired benefits of the intervention is not well understood. A challenge to understanding the impact of specific types of modifications is a lack of attention to characterizing the different types of changes that may occur. A system for classifying the types of modifications that are made when interventions and programs are implemented can facilitate efforts to understand the nature of modifications that are made in particular contexts as well as the impact of these modifications on outcomes of interest.MethodsWe developed a system for classifying modifications made to interventions and programs across a variety of fields and settings. We then coded 258 modifications identified in 32 published articles that described interventions implemented in routine care or community settings.ResultsWe identified modifications made to the content of interventions, as well as to the context in which interventions are delivered. We identified 12 different types of content modifications, and our coding scheme also included ratings for the level at which these modifications were made (ranging from the individual patient level up to a hospital network or community). We identified five types of contextual modifications (changes to the format, setting, or patient population that do not in and of themselves alter the actual content of the intervention). We also developed codes to indicate who made the modifications and identified a smaller subset of modifications made to the ways that training or evaluations occur when evidence-based interventions are implemented. Rater agreement analyses indicated that the coding scheme can be used to reliably classify modifications described in research articles without overly burdensome training.ConclusionsThis coding system can complement research on fidelity and may advance research with the goal of understanding the impact of modifications made when evidence-based interventions are implemented. Such findings can further inform efforts to implement such interventions while preserving desired levels of program or intervention effectiveness.

[1]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[2]  G. Gopalan,et al.  Best practices: wellness self-management: an adaptation of the illness management and recovery program in New York State. , 2011, Psychiatric services.

[3]  R. Jacobs,et al.  Cultural adaptation in translational research: Field experiences , 2005, Journal of Urban Health.

[4]  A. Lau.  Making the Case for Selective and Directed Cultural Adaptations of Evidence-Based Treatments: Examples From Parent Training , 2006 .

[5]  Felipe González Castro,et al.  The Cultural Adaptation of Prevention Interventions: Resolving Tensions Between Fidelity and Fit , 2004, Prevention Science.

[6]  G. Gopalan,et al.  Wellness Self-Management: An Adaptation of the Illness Management and Recovery Practice in New York State , 2011 .

[7]  K. Hofman,et al.  Implementation Science , 2007, Science.

[8]  R. Remien,et al.  Moving From Theory to Research to Practice: Implementing an Effective Dyadic Intervention to Improve Antiretroviral Adherence for Clinic Patients , 2006, Journal of acquired immune deficiency syndromes.

[9]  Delbert S. Elliott,et al.  Issues in Disseminating and Replicating Effective Prevention Programs , 2004, Prevention Science.

[10]  Keith Zvoch.  Treatment Fidelity in Multisite Evaluation , 2009 .

[11]  C. Webster-Stratton,et al.  Adapting The Incredible Years, an evidence‐based parenting programme, for families involved in the child welfare system , 2010 .

[12]  Redford B Williams,et al.  Adaptation and implementation of an evidence-based behavioral medicine program in diverse global settings: The Williams LifeSkills experience , 2011, Translational behavioral medicine.

[13]  R. Dushay,et al.  LOWERING HIV RISK AMONG ETHNIC MINORITY DRUG USERS: COMPARING CULTURALLY TARGETED INTERVENTION TO A STANDARD INTERVENTION , 2001, The American journal of drug and alcohol abuse.

[14]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[15]  H. Ireys,et al.  Reinventing fidelity: The transfer of social technology among settings , 1991, American journal of community psychology.

[16]  Christopher J. Miller,et al.  Community mental health provider modifications to cognitive therapy: implications for sustainability. , 2013, Psychiatric services.

[17]  Henna Hasson,et al.  Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care , 2012, Implementation Science.

[18]  M. Shediac-Rizkallah,et al.  Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. , 1998, Health education research.

[19]  R. Malow,et al.  Context and common ground: cultural adaptation of an intervention for minority HIV infected individuals. , 2004, Journal of cultural diversity.

[20]  Culturally tailored HIV-AIDS risk-reduction messages targeted to African-American urban women: impact on risk sensitization and risk reduction. , 1993, Journal of consulting and clinical psychology.

[21]  D. Hinton,et al.  CBT for Vietnamese Refugees with Treatment-Resistant PTSD and Panic Attacks: A Pilot Study , 2004, Journal of traumatic stress.

[22]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[23]  R. Noonan,et al.  Adoption, Adaptation, and Fidelity of Implementation of Sexual Violence Prevention Programs , 2009, Health promotion practice.

[24]  B. Stanton,et al.  The complex business of adapting effective interventions to new populations: an urban to rural transfer. , 2005, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[25]  M. A. Scheirer,et al.  An agenda for research on the sustainability of public health programs. , 2011, American journal of public health.

[26]  L. L. McIntyre.  Adapting Webster-Stratton's incredible years parent training for children with developmental delay: findings from a treatment group only study. , 2008, Journal of intellectual disability research : JIDR.

[27]  A. Strauss,et al.  Basics of Qualitative Research , 1992 .

[28]  L. Hill,et al.  A Focused Approach to Assessing Program Fidelity , 2007, Prevention Science.

[29]  D. Kaysen,et al.  Psychological Trauma: Theory, Research, Practice, and Policy Adaptation of Cognitive Processing Therapy for Treatment of Torture Victims: Experience in Kurdistan, Iraq , 2011 .

[30]  K. Varjas,et al.  Participatory Model of Mental Health Programming: Lessons Learned from Work in a Developing Country. , 1998 .

[31]  Aaron R. Lyon,et al.  Easier Said than Done: Intervention Sustainability in an Urban After-School Program , 2011, Administration and Policy in Mental Health and Mental Health Services Research.

[32]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[33]  K. Kumpfer,et al.  Cultural Sensitivity and Adaptation in Family-Based Prevention Interventions , 2002, Prevention Science.

[34]  J. Unützer,et al.  Project IMPACT: A Report on Barriers and Facilitators to Sustainability , 2006, Administration and Policy in Mental Health and Mental Health Services Research.

[35]  Christine M Markham,et al.  Using Intervention Mapping to Adapt an Effective HIV, Sexually Transmitted Disease, and Pregnancy Prevention Program for High-Risk Minority Youth , 2005, Health promotion practice.

[36]  J. Lochman,et al.  Guidelines for Adapting Manualized Interventions for New Target Populations: A Step-Wise Approach Using Anger Management as a Model. , 2012, Clinical psychology : a publication of the Division of Clinical Psychology of the American Psychological Association.

[37]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[38]  Stacy L. Frazier,et al.  Toward the Effective and Efficient Measurement of Implementation Fidelity , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[39]  L. Palinkas,et al.  Dynamic adaptation process to implement an evidence-based child maltreatment intervention , 2012, Implementation Science.

[40]  M. A. Scheirer.  Linking sustainability research to intervention types. , 2013, American journal of public health.

[41]  M. Amodeo,et al.  Modifications of evidence-based practices in community-based addiction treatment organizations: a qualitative research study. , 2011, Addictive behaviors.

[42]  Adapting an evidence-based intervention: tales of the Hustler Project. , 2003, AIDS education and prevention : official publication of the International Society for AIDS Education.

[43]  C. Webster-Stratton,et al.  Adapting the Incredible Years child dinosaur social, emotional, and problem-solving intervention to address comorbid diagnoses , 2008 .

[44]  Paul N Williams,et al.  Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. , 2000, AIDS education and prevention : official publication of the International Society for AIDS Education.

[45]  J. Owczarzak,et al.  Provider perspectives on evidence-based HIV prevention interventions: barriers and facilitators to implementation. , 2011, AIDS patient care and STDs.

[46]  Chris Melde,et al.  Addressing Program Fidelity Using Onsite Observations and Program Provider Descriptions of Program Delivery , 2006, Evaluation review.

[47]  N. Jacobson,et al.  Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. , 1993, Journal of consulting and clinical psychology.

[48]  M. Kennedy,et al.  The effect of tailoring a model HIV prevention program for local adolescent target audiences. , 2000, AIDS education and prevention : official publication of the International Society for AIDS Education.

[49]  A. Garland,et al.  The GANA Program: A Tailoring Approach to Adapting Parent Child Interaction Therapy for Mexican Americans , 2005 .

[50]  T. Galovski,et al.  Manualized therapy for PTSD: flexing the structure of cognitive processing therapy. , 2012, Journal of consulting and clinical psychology.

[51]  G. Aarons,et al.  Adaptation happens: a qualitative case study of implementation of The Incredible Years evidence‐based parent training programme in a residential substance abuse treatment programme , 2012 .

[52]  J. Kimberly,et al.  The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research , 2012, Implementation Science.

[53]  Loretta S. Malta,et al.  The flexible application of a manualized treatment for PTSD symptoms and functional impairment related to the 9/11 World Trade Center attack. , 2007, Behaviour research and therapy.

[54]  Rona Campbell,et al.  High fidelity? How should we consider variations in the delivery of school-based health promotion interventions? , 2009 .

[55]  K. Varjas,et al.  The Participatory Intervention Model: A Framework for Conceptualizing and Promoting Intervention Acceptability , 2000 .

[56]  Mathea Falco,et al.  A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. , 2003, Health education research.

[57]  The World Starts With Me: using intervention mapping for the systematic adaptation and transfer of school-based sexuality education from Uganda to Indonesia , 2011, Translational behavioral medicine.