AC 2007-2343: ASSESSMENTS FOR THREE PERFORMANCE AREAS IN CAPSTONE ENGINEERING DESIGN

Capstone engineering design courses occupy pivotal positions in every engineering baccalaureate degree program. They are critical to preparing graduates with professional skills needed for innovative, responsible practice in a global environment, and they provide vital assessment data for ABET accreditation of degree programs. This paper describes assessment instruments developed for capstone engineering design courses, filling a crucial gap facing design educators. Seven assessment exercises are presented to address three areas of performance for capstone engineering design. Each exercise is accompanied by a scoring rubric structured around performance factors and five levels of performance. Suggestions given for utilization for formative and summative purposes make these assessments valuable for guiding student learning and assigning performance scores or grades. These assessments constitute foundational parts of an assessment system for capstone engineering design courses. Introduction National leaders have called for reform of engineering education to prepare engineering graduates for the competitive global market place 1, 2 . Among capabilities cited as deficient in student preparation are professional skills and abilities to innovate technical products in the context of business conditions 3, 4 . Oftentimes, these topics are not given appropriate attention in engineering programs. Important professional skill development is often assigned to capstone engineering design courses. These courses are the culminating experiences for undergraduate engineering students, and they often incorporate client-driven design projects that have significant professional challenges. Surveys of capstone design instructors, however, indicate that instructional focus and assessment of student learning vary greatly, depending on instructor preferences and abilities. Seldom are comprehensive outcomes defined for these courses 5 . This raises concerns regarding student preparation and ABET accreditation, both of which require assessment of design and many professional outcomes 6 . More than a decade ago, Richard Stiggins made a strong case for classroom assessment as the cornerstone to effective instruction 7 . He argues that with clear achievement targets and appropriate assessment strategies, students are more likely to increase their achievement since they understand what is expected of them. In addition, Black and Wiliam 8 , studying classroom assessment practices across grades, disciplines, and countries, documented overwhelming evidence that classroom assessment can enhance student achievement and academic well being. Therefore, high quality classroom assessment in capstone design courses is vital due to pivotal roles that capstone design courses play in engineering curricula. Also, because these courses are a required part of accredited engineering programs in the US, they provide a rich environment for assessing a variety of student learning outcomes and associated program achievements. A number of researchers report strategies for assessing student performance in capstone engineering design courses 9-13 . Some instructors assess student work in capstone design courses by focusing more on design steps than on the quality of design products 5 , while others focus on design products with little attention given to design processes 14 . However, success after graduation requires engineers to produce high quality products while also refining their processes and developing professionally to support continuous improvement of product quality. This argues the importance of capstone design course outcomes related to both student learning and product development. The authors of this paper have a record of developing instructional materials and assessments for engineering design. As part of the TIDEE {Transferable Integrated Design Engineering Education] consortium, they have defined design learning outcomes based on surveys of capstone design instructors and industry representatives 4, 15 . They also defined design learning outcomes and created assessments for the first two years of engineering design curricula 16 . Currently, they are leading a National Science Foundation project, “Transferable Assessments for Capstone Engineering Design” which is developing and testing research-based assessments for capstone design courses across disciplines and institutions. In this process, they found that few engineering faculty use proven assessments for formative assessment in capstone design courses 17 . This finding justified development of a cognitive development model for capstone design 18 and a framework for guiding development of sound assessments for capstone design courses 19 . Goal The goal of the NSF project, “Transferable Assessments for Capstone Engineering Design,” is to produce versatile, sound classroom assessment instruments for assessing student achievement in capstone engineering design courses. The goal of this paper is to present the assessments produced by this project. This paper presents a set of three assessments, comprised of specific assessment exercises and corresponding scoring rubrics based on the cognitive development model and assessment framework established earlier. The paper also offers suggestions for use of assessments in formative and summative applications. Areas of Performance Capstone design courses must produce capable students and quality design products. Researchers report that design performances of experts are evidenced by their thought processes, personal abilities, understanding of important requirements, and capabilities to deliver valuable products. The authors, with the help of design education consultants, defined four areas of performance that encompass essential aspects of engineering design performances: Learner development outcomes are defined under two areas of performance: (1) Personal capacity: Individuals performing and improving individual skills essential to engineering design (2) Team processes: Teams developing and implementing collective processes that support team productivity in design Solution development outcomes are defined under two additional areas of performance: (3) Solution requirements: Definition of targeted design solution performance and features expected to satisfy stakeholder needs and constraints (4) Solution assets: Results from a design project that meet needs and deliver satisfaction and value to key project stakeholders For each of these areas, performance criteria were defined to articulate performances desired in a capstone engineering design course. These definitions embrace the critical factors relevant to each of these areas of performance, so they also suggest a framework for measuring achievement. Performance criteria and corresponding factors identified for each performance area are listed below.