Fidelity of Implementation in Project Towards No Drug Abuse (TND): A Comparison of Classroom Teachers and Program Specialists

This paper presents the results of an effectiveness trial of Project Towards No Drug Abuse [TND], in which we compared program delivery by regular classroom teachers and program specialists within the same high schools. Within 18 schools that were randomly assigned to the program or control conditions, health classrooms were assigned to program delivery by teachers or (outside) specialists. Classroom sessions were observed by pairs of observers to assess three domains of implementation fidelity: adherence, classroom process, and perceived student acceptance of the program. Pre- and immediate posttest survey data were collected from 2331 students. Of the four composite indexes of implementation fidelity that were examined, only one (quality of delivery) showed a difference between specialists and teachers, with marginally higher ratings of specialists (p < .10). Both teachers and program specialists achieved effects on three of the five immediate outcome measures, including program-specific knowledge, addiction concern, and social self-control. Students’ posttest ratings of the program overall and the quality of program delivery failed to reveal differences between the teacher- and specialist-led classrooms. These results suggest that motivated, trained classroom teachers can implement evidence-based prevention programs with fidelity and achieve immediate effects.

[1]  G. Clarke Improving the transition from basic efficacy research to effectiveness studies: methodological issues and procedures. , 1995, Journal of consulting and clinical psychology.

[2]  C. Dent,et al.  The correlates of addiction concern among adolescents at high risk for drug abuse. , 1996, Journal of substance abuse.

[3]  C. Dent,et al.  Implementation and Process Evaluation of a Student "School-as-Community" Group , 1997, Evaluation review.

[4]  K. Brown,et al.  Effectiveness of a social influences smoking prevention program as a function of provider type, training method, and school risk. , 1999, American journal of public health.

[5]  J. Graham,et al.  Program integrity as a moderator of prevention program effectiveness: results for fifth-grade students in the adolescent alcohol prevention trial. , 1991, Journal of studies on alcohol.

[6]  William B Hansen,et al.  How all Stars Works: An Examination of Program Effects on Mediating Variables , 2004, Health education & behavior : the official publication of the Society for Public Health Education.

[7]  C. Dent,et al.  The associations of social self-control, personality disorders, and demographics with drug use among high-risk youth. , 2003, Addictive behaviors.

[8]  S. Schoenwald,et al.  Effectiveness, transportability, and dissemination of interventions: what matters when? , 2001, Psychiatric services.

[9]  T. Baranowski,et al.  How best to measure implementation of school health curricula: a comparison of three measures. , 1998, Health education research.

[10]  R. Glasgow,et al.  Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. , 2003, American journal of public health.

[11]  Louise A. Rohrbach,et al.  Diffusion of School-Based Substance Abuse Prevention Programs , 1996 .

[12]  Deborah Bybee,et al.  Fidelity Criteria: Development, Measurement, and Validation , 2003 .

[13]  Mathea Falco,et al.  A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. , 2003, Health education research.

[14]  Louise A. Rohrbach,et al.  Case Study 7: Chapter 17: Development and Pilot Testing of Project SMART , 2001 .

[15]  N. Tobler,et al.  Effectiveness of School-Based Drug Prevention Programs: A Meta-Analysis of the Research , 1997, Journal of Primary Prevention.

[16]  W. B. Hansen,et al.  Drug education practice: results of an observational study. , 1999, Health education research.

[17]  J. Graham,et al.  Reliability of Self-Report Measures of Drug Use in Prevention Research: Evaluation of the Project Smart Questionnaire via the Test-Retest Reliability Matrix , 1984, Journal of drug education.

[18]  T. Wills,et al.  The Motivation, Skills, and Decision-Making Model of “Drug Abuse” Prevention , 2004, Substance use & misuse.

[19]  G. Botvin,et al.  Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: results of a 3-year study. , 1990, Journal of consulting and clinical psychology.

[20]  J. Nunnally Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[21]  D. Weiss,et al.  Interrater reliability and agreement of subjective judgments , 1975 .

[22]  C. Dent,et al.  Tobacco, Alcohol, and other Drug use among High-Risk Young People: A Five-Year Longitudinal Study from Adolescence to Emerging Adulthood , 2005 .

[23]  Steven Yale Sussman,et al.  Handbook of Program Development for Health Behavior Research and Practice , 2001 .

[24]  G. Botvin,et al.  Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. , 1995, JAMA.

[25]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[26]  Mark T. Greenberg,et al.  The Study of Implementation: Current Findings From Effective Programs that Prevent Mental Disorders in School-Aged Children , 2000 .

[27]  S. Sussman,et al.  An Evaluation of the Fidelity of Implementation of a School-Based Drug Abuse Prevention Program: Project toward No Drug Abuse (TND) , 2005, Journal of drug education.

[28]  N. Tobler,et al.  School-Based Adolescent Drug Prevention Programs: 1998 Meta-Analysis , 2000, Journal of Primary Prevention.

[29]  C. Dent,et al.  Project Towards No Drug Abuse: long-term substance use outcomes evaluation. , 2006, Preventive medicine.

[30]  C. Dent,et al.  Implementation and Process Evaluation of a School-Based Drug Abuse Prevention Program: Project towards No Drug Abuse , 1998, Journal of drug education.

[31]  C. Dent,et al.  Project Towards No Drug Abuse , 2000 .

[32]  B. Flay Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. , 1986, Preventive medicine.

[33]  M. J. Lewis,et al.  The effect of two types of teacher training on implementation of Smart Choices: a tobacco prevention curriculum. , 1994, The Journal of school health.

[34]  Rudolf H. Moos,et al.  Evaluating Educational Environments , 1979 .

[35]  D. Osgood,et al.  Results of an Independent Evaluation of Project ALERT Delivered in Schools by Cooperative Extension , 2005, Prevention Science.

[36]  C. Perry,et al.  Evaluating the statewide dissemination of smoking prevention curricula: factors in teacher compliance. , 1990, The Journal of school health.

[37]  B. Flay,et al.  Developing School-Based Tobacco Use Prevention and Cessation Programs , 1994 .

[38]  Steven M. Giles,et al.  Evaluation of the All Stars Character Education and Problem Behavior Prevention Program: Effects on Mediator and Outcome Variables for Middle School Students , 2001, Health education & behavior : the official publication of the Society for Public Health Education.

[39]  P. Greenwald,et al.  The new emphasis in cancer control. , 1985, Journal of the National Cancer Institute.

[40]  G. Botvin,et al.  School-based smoking prevention: the teacher training process. , 1989, Preventive medicine.

[41]  G. Botvin,et al.  Smoking prevention among urban minority youth: assessing effects on outcome and mediating variables. , 1992, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[42]  C. Dent,et al.  Project towards no drug abuse: a review of the findings and future directions. , 2002, American Journal of Health Behavior.

[43]  J. Graham,et al.  Diffusion of a school-based substance abuse prevention program: predictors of program implementation. , 1993, Preventive medicine.

[44]  D M Murray,et al.  Planning for the appropriate analysis in school-based drug-use prevention studies. , 1990, Journal of consulting and clinical psychology.

[45]  David B. Wilson,et al.  Characteristics of Effective School-Based Substance Abuse Prevention , 2003, Prevention Science.

[46]  Andrew V. Dane,et al.  Program integrity in primary and early secondary prevention: are implementation effects out of control? , 1998, Clinical psychology review.

[47]  Bruce Joyce,et al.  Improving Inservice Training: The Messages of Research. , 1980 .

[48]  Russell E. Glasgow,et al.  Evaluating the Relevance, Generalization, and Applicability of Research , 2006, Evaluation & the health professions.

[49]  T. Valente,et al.  Type II Translation , 2006, Evaluation & the health professions.

[50]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.