Long Term Effects Of A Middle School Engineering Outreach Program For Girls: A Controlled Study

This study compares the high school choices and choice of college major of two groups: young women who participated in the two-week Camp Reach engineering program as rising sixth graders, and those who applied to the program but were not chosen in the random lottery (control group). Results indicate that, in comparison to the control group, Camp Reach participants were significantly more likely to attend a public high school specializing in mathematics and science and also more likely to enroll in elective math and science courses in high school. While a higher fraction of the Camp Reach group chose engineering majors upon college entry, the difference did not reach statistical significance. Grouping all STEM-related majors together, choices of the Camp Reach and control groups were not significantly different. Furthermore, there were no significant differences in the engineering self-efficacy and other measures of efficacy between the Camp Reach and control groups. Introduction and Background The crisis of under-representation of women in engineering continues unabated and in fact is projected to be worsening. Enrollment statistics compiled by the American Society for Engineering Education (ASEE) for the period 1999-2005 show only a small increase in the number of B.S. engineering degrees awarded to women, insufficient to meet workforce demands. Furthermore, the number of women enrolling in engineering programs decreased in the 2004-05 academic year. Among a variety of strategies being employed by universities, summer outreach programs for girls and/or other minorities, also referred to as pipeline or intervention programs, are relatively common. In their study of the impact of the National Science Foundation’s Program for Women and Girls, Darke, Clewell, and Sevo found evidence that summer camps were “successful in achieving positive change” for girls in science, technology, engineering, and mathematics (STEM). Still, however, there are key gaps in evaluation of these programs that limit the extent to which their value and cost/benefit proposition can be judged or compared to those of other strategies. One limitation of existing summative program evaluation is that typically only short-term impacts are measured, comparing pre-program and post-program data on participants’ attitudes about and knowledge of engineering. While widening the pipeline is the fundamental mission of most programs, there is very little knowledge of the extent to which short-term positive effects are sustained and realized over the long term. One conclusion of the review by Darke, Clewell, and Sevo was a call for longitudinal studies: There is a need for innovative methods to encourage the assessment of longitudinal outcomes. Many programs are unable to assess the long-term outcomes of their efforts because of the difficulty and expense of collecting outcome data on participants over a period of several years.... The development of innovative approaches to facilitate the collection of longitudinal data would make a significant contribution to program evaluation. (p. 300) This challenge of longitudinal evaluation is exacerbated for middle school programs. Many researchers have argued that early interventions, prior to high school, are critical, yet there is such a long lag time between the intervention and the point of college entry that longitudinal evaluation is especially challenging. Another limitation of prior research on the effects of STEM intervention programs that is rarely acknowledged has been the inability to eliminate or control for the self-selection factor. In other words, it is possible or probable that girls who choose to participate in an engineering outreach program, even in middle school, are those who would be inclined to choose a STEM major and career even without participation in the program. Appropriate control or comparison groups can be difficult or impossible to identify and engage. This study closes both of these gaps in prior evaluation of pipeline programs for girls. In this paper we report the initial findings of an ongoing longitudinal evaluation of Camp Reach, a summer engineering camp for girls entering the 7 grade. The Camp Reach admissions process was specifically designed to enable a control group, so that self-selection could be eliminated as an explanation for any long-term impacts. After ten years in operation there are now four camper cohorts and associated control groups that have graduated from high school. From them we have collected information about their high school experience, knowledge of engineering, selfefficacy beliefs, and initial choice of college major. To the best of our knowledge, this study is the first longitudinal evaluation of a middle school intervention program that has used this type of controlled research design.