Program Provider and Observer Ratings of School-Based Preventive Intervention Implementation: Agreement and Relation to Youth Outcomes

Few prevention studies have examined the degree to which different measures of program implementation adherence predict youth outcomes. The current study was conducted with rural middle school youth participating in a longitudinal school-based preventive intervention program. Study participants’ average age at the pretest assessment was 12.3 years. The association between program implementation ratings supplied by provider self-reports and trained independent observer reports were evaluated. In addition, the relationship between measures of implementation and youth outcomes were examined. Results indicated that although program providers tended to report higher implementation than independent observers, most ratings were correlated significantly across raters. Observer-reported implementation ratings significantly predicted several youth substance-related outcomes, while provider-reported self-ratings did not.Program provider characteristics predicted several youth outcomes. Findings suggest that there might be a social desirability bias in provider self-reported ratings of implementation and that caution must be used when interpreting self-reported ratings of implementation.

[1]  G. Botvin,et al.  Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. , 1995, JAMA.

[2]  William B. Hansen,et al.  Effects of Program Implementation on Adolescent Drug Use Behavior , 1990 .

[3]  Neal Schmitt,et al.  The fidelity-adaptation debate: Implications for the implementation of public sector social programs , 1987 .

[4]  Eva Lantos Rezmovic,et al.  Measuring the Degree of Program Implementation , 1983 .

[5]  G. Botvin.  Life skills training : promoting health and personal development , 1998 .

[6]  R. Luepker,et al.  Teenage health teaching modules: impact of teacher training on implementation and student outcomes. , 1991, The Journal of school health.

[7]  K. Kumpfer,et al.  Family-Oriented Interventions for the Prevention of Chemical Dependency in Children and Adolescents. , 1986 .

[8]  J. Elder,et al.  Correlates of health facilitator performance in a tobacco use prevention program: implications for recruitment. , 1990, The Journal of school health.

[9]  R. Spoth,et al.  The Strengthening Families Program for the prevention of delinquency and drug use. , 1996 .

[10]  M. Guyll,et al.  Two studies of proximal outcomes and implementation quality of universal preventive interventions in a community–university collaboration context , 2002 .

[11]  D. Connell,et al.  Summary of findings of the School Health Education Evaluation: health promotion effectiveness, implementation, and costs. , 1985, The Journal of school health.

[12]  Andrew V. Dane,et al.  Program integrity in primary and early secondary prevention: are implementation effects out of control? , 1998, Clinical psychology review.

[13]  K. Kumpfer,et al.  Competency Training The Strengthening Families Program : For Parents and Youth 10 – 14 , 2000 .

[14]  B. Flay.  Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. , 1986, Preventive medicine.

[15]  R. Spoth.  Challenges in Defining and Developing the Field of Rural Mental Disorder Preventive Intervention Research , 1997, American journal of community psychology.

[16]  Joseph A. Durlak,et al.  Successful Prevention Programs for Children and Adolescents , 1997 .

[17]  Guy S. Parcel,et al.  Beyond demonstration: Diffusion of health promotion innovations. , 1990 .

[18]  Kenneth W Griffin,et al.  Drug Abuse Prevention Among Minority Adolescents: Posttest and One-Year Follow-Up of a School-Based Preventive Intervention , 2001, Prevention Science.

[19]  J. Graham,et al.  Diffusion of a school-based substance abuse prevention program: predictors of program implementation. , 1993, Preventive medicine.

[20]  Joseph A. Durlak,et al.  Why Program Implementation is Important , 1998 .

[21]  Gene E. Hall,et al.  A Developmental Model for Determining Whether the Treatment is Actually Implemented , 1977 .

[22]  G. Botvin,et al.  School-based smoking prevention: the teacher training process. , 1989, Preventive medicine.

[23]  S. W. Fors,et al.  Implication of Results: Factors for Success. , 1985 .

[24]  Huey-tsyh Chen.  Theory-driven evaluations , 1990 .

[25]  C. Basch,et al.  Avoiding Type III Errors in Health Education Program Evaluations: A Case Study , 1985, Health education quarterly.

[26]  G. Botvin,et al.  Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: results of a 3-year study. , 1990, Journal of consulting and clinical psychology.

[27]  R. Selman,et al.  “I feel I have received a new vision:” An analysis of teachers' professional development as they work with students on interpersonal issues , 1997 .

[28]  B. Flay,et al.  The integrity of smoking prevention curriculum delivery , 1989 .

[29]  A. Bandura.  Social Foundations of Thought and Action: A Social Cognitive Theory , 1985 .

[30]  Mark T. Greenberg,et al.  The Study of Implementation: Current Findings From Effective Programs that Prevent Mental Disorders in School-Aged Children , 2000 .

[31]  Susan Heck,et al.  Measuring Innovation Configurations: Procedures and Applications. , 1981 .

[32]  E. Labouvie.  Problem Behavior and Psychosocial Development; a Longitudinal Study of Youth , 1978 .