AC 2008-906: ACADEMIC PATHWAYS STUDY: PROCESSES AND REALITIES

Amid concerns that U.S. educational institutions are not attracting and graduating sufficient numbers of science, technology, engineering and mathematics (STEM) students with the skills and knowledge needed to tackle the technological challenges of the 21 century, the National Science Foundation granted funding in 2003 to the Center for the Advancement of Engineering Education (CAEE), dedicated to advancing the scholarship of engineering learning and teaching. The largest element of the CAEE is the Academic Pathways Study (APS), an in-depth, mixed methods exploration of the undergraduate student experience and the graduate’s transition into professional practice. The APS addresses the following research questions: 1. How do students' engineering skills and knowledge develop and/or change over time? 2. How does one's identity as an engineer evolve? 3. What elements of engineering education contribute to the students' skills/knowledge and identity? What elements contribute to students’ persistence in an engineering major and persistence in the engineering profession? 4. What skills do early career engineers need as they enter the workplace? Given the scale of the APS investigation with multiple schools and student populations, the answers to these questions will allow us to identify educational practices that contribute to students persisting and thriving in engineering, and potential strategies for attracting more students to the study of engineering. This paper describes the evolution and implementation of the Academic Pathways Study (APS), a five year, multi-institution study designed to address these questions and implications for academic practices. As such, this paper is a “welcome mat” or introduction for those interested in learning more about APS. Components of the paper address questions researchers designing new studies may have about the organizational and technical infrastructure that supported this project, or about the quantitative and qualitative research methods, tools, and protocols used. Other components of the paper address questions that researchers and engineering faculty and administrators might have regarding how to explore the findings and insights that are emerging from this extensive longitudinal and cross-sectional study of students’ pathways through engineering. Research findings to date are summarized in a companion paper entitled Findings from the Academic Pathways Study of Engineering Undergraduates, by Atman, et al. 1. APS Background and Goals The past two decades have witnessed an ongoing national dialog about the lack of gender, race and ethnic diversity among those studying and practicing engineering and the adequacy of students’ preparation for today’s engineering challenges. Further complicating the discussions are worries that U.S. educational institutions are not attracting and retaining sufficient students in the science, technology, engineering and math (STEM) fields to keep up with the country's demands. In response, the National Science Foundation set out in 2002 to establish Higher P ge 13137.3 Education Centers to promote exemplary education in these fields. One of the centers created by NSF is the Center for the Advancement of Engineering Education (CAEE). CAEE consists of three research elements: Scholarship on Learning Engineering, Scholarship on Teaching Engineering, and the Institute for Scholarship on Engineering Education. These elements bring together a team of scholars and experts from an array of backgrounds, disciplines and universities to collaboratively accomplish the mission of improving the knowledge and practice of engineering teaching and learning. This paper focuses on a major undertaking of CAEE’s Scholarship on Learning element, the Academic Pathways Study (APS). The paper begins by situating the APS in the existing knowledge base of engineering education and goes on to describe the study’s organization and execution, starting with the research team and leadership, followed by the study design, research cohorts and methods. The paper closes with a discussion of the research challenges, implications for engineering education, and possible future research. APS results are reported in separate papers as they become available, with findings to date summarized by Atman, et al. The Academic Pathways Study aims to improve educational effectiveness by developing a rich understanding of the engineering student experience. To that end, APS addresses the following student-centric research questions: 1. How do students' engineering skills and knowledge develop and/or change over time? 2. How does one's identity as an engineer evolve? 3. What elements of engineering education contribute to the students' skills/knowledge and identity? What elements contribute to students’ persistence in an engineering major and persistence in the engineering profession? 4. What skills do early career engineers need as they enter the workplace? The first of these questions addresses cognitive outcomes, the second deals with affective learning, and the third cluster of questions examines the interplay of outcomes and environmental factors critical to student success. Taken together, these questions align with the highly durable and influential input-environment-outcome (I-E-O) model of college impact, first proposed by Alexander Astin over thirty years ago. Furthermore, the APS research questions seek to explore the evolutionary nature of outcomes and environmental influences, tracing their development and change over time. Inherent in the research questions is the anticipation that the study will generate recommendations for improving educational practices to enhance the student experience and persistence in engineering studies, as well as suggesting potential strategies for attracting more students to the discipline. This is certainly not the first study of the engineering student experience; there is solid prior work to build on. A few of the studies that have influenced and informed the APS design deserve note. Seymour and Hewitt conducted a three-year study of 460 students at seven institutions, investigating why students leave or persist in science, mathematics and engineering (SME) majors. Using ethnographic interviews, Seymour and Hewitt studied attrition among SME majors, with the aim of deriving a set of testable hypotheses from student reflections. They P ge 13137.4 evaluated how students weighed numerous factors in deciding to leave SME for non-SME majors or, conversely, to persist in SME majors despite challenges and setbacks. Seymour and Hewitt's work suggests that students are leaving engineering not for lack of ability, but because of structural and cultural factors such as inadequate teaching, overly competitive grading, and lack of identification with the associated careers. Seymour and Hewitt’s findings illustrate the complex nature of deciding to study or not study in SME fields, leading APS researchers to include a broad range of questions and prompts in APS research tools so as to not prescribe responses. Astin’s research on student development in higher education relied on large-scale surveys conducted with over 200,000 students. His surveys of first-year and fourth-year students over a twenty-year period led Astin to conclude that the level of student involvement is directly proportional to student learning. Astin defines student involvement as the amount of physical and psychological energy devoted by a student to the academic experience. An environmental factor that Astin identifies as being highly influential is the student’s major. He concludes (page 371) that “Engineering produces more significant effects on student outcomes than any other major field.” Majoring in engineering was positively correlated with the development of strong analytic skills (page 237) and job-related skills (page 240); it was negatively correlated with overall satisfaction with the college experience, satisfaction with curriculum and instruction, and developing a diversity orientation (page 306). Astin’s findings led APS researchers to design a study that examines the effect of an engineering major over time, looks at engineering students relative to others, and considers a variety of institutional factors. Adelman studied engineering undergraduate careers by drawing evidence from the 11-year college transcript history of the High School & Beyond/Sophomore Cohort Longitudinal Study, as well as the high school transcripts, test scores and surveys of this nationally representative sample. Adelman introduces the idea of curricular momentum, which can reinforce student trajectories within engineering and establish preferred pathways for students leaving engineering, as well as boundaries for students who might be interested in entering the engineering field. Adelman's work shows the importance of curricular factors in influencing how students explore and choose majors. His findings illustrate the need to have enough fidelity in research instruments to capture the subtle dimensions of navigating and defining an academic pathway. These studies and others–such as the National Survey of Student Engagement (NSSE) 10 and the Pittsburgh Freshman Survey–along with the expertise of APS researchers, suggested that multiple methods and multiple student cohorts were needed to fully capture the engineering student experience. To this end, the APS research design included: • Both qualitative and quantitative methods. Qualitative methods allow for exploration of how students arrive at the decision to major in engineering and how they navigate the educational process, whereas quantitative methods elicit information from larger numbers of students on a broad, but defined range of issues, such as degree of academic engagement, perceptions and attitudes about engineering, and motivations for pursuing an engineering major. • Multiple student cohorts across multiple institutions to explore the overar