The Development and Evaluation of DEFT, a Web-Based Tool for Engineering Design Education

This article describes the development of the Design Evaluation and Feedback Tool (DEFT), a custom-built web-based system that collects and reports data to support teaching, learning, and research in project-based engineering design education. The DEFT system collects data through short weekly questionnaires for students and instructors in engineering design classes, and uses these data to produce weekly reports for both types of user. The system is intended to engage students in reflective reporting on their experiential learning, to support educators in coaching student designers, and to serve as a data collection tool for education researchers. DEFT was developed through an iterative design and evaluation process, involving 185 students and 18 instructors at two universities. The system was evaluated using a combination of participation observation, user interviews, and anonymous questionnaires, and the results guided subsequent improvements to the system. This article describes the development and evaluation process, provides an overview of the resulting system, and ends by discussing the potential for DEFT to be used in evaluating and improving project-based design classes.

[1]  Katherine K. Perkins,et al.  Modifying and Validating the Colorado Learning Attitudes about Science Survey for Use in Chemistry , 2008 .

[2]  Gerd Fricke,et al.  Successful individual approaches in engineering design , 1996 .

[3]  Jami J. Shah,et al.  Applied Tests of Design Skills—Part II: Visual Thinking , 2013 .

[4]  Peter Lloyd,et al.  Ethnographic description of design networks , 1998 .

[5]  J. Dewey.  Experience and Education , 1938 .

[6]  Carol R. Rodgers,et al.  Defining Reflection: Another Look at John Dewey and Reflective Thinking , 2002, Teachers College Record: The Voice of Scholarship in Education.

[7]  D. Kolb.  Experiential Learning: Experience as the Source of Learning and Development , 1983 .

[8]  Oenardi Lawanto,et al.  Advancing research on engineering design using e-Journal , 2015, 2015 IEEE Frontiers in Education Conference (FIE).

[9]  Josip Slisko,et al.  Software Socrative and Smartphones as Tools For Implementation of Basic Processes of Active Physics Learning in Classroom: An Initial Feasibility Study With Prospective Teachers , 2013 .

[10]  Linden J. Ball,et al.  Structured and opportunistic processing in design: a critical discussion , 1995, Int. J. Hum. Comput. Stud..

[11]  John S. Gero,et al.  Comparing collaborative co-located and distributed design processes in digital and traditional sketching environments: A protocol study using the function-behaviour-structure coding scheme , 2011 .

[12]  Nigel Cross,et al.  Forty years of design research , 2007 .

[13]  Nicole Clark,et al.  Self and Peer Assessment in Software Engineering Projects , 2005, ACE.

[14]  Louis L. Bucciarelli,et al.  An ethnographic perspective on engineering design , 1988 .

[15]  Ben J Hicks,et al.  Activity Theory as a means for multi-scale analysis of the engineering design process: A protocol study of design in practice , 2015 .

[16]  Matthew W. Ohland,et al.  Development of a Theory-Based Assessment of Team Member Effectiveness , 2007 .

[17]  Thomas J. Howard,et al.  Describing the creative design process by the integration of engineering design and cognitive psychology literature , 2008 .

[18]  Diana E. Forsythe,et al.  Using ethnography in the design of an explanation system , 1995 .

[19]  Raymonde Guindon.  Designing the design process: exploiting opportunistic thoughts , 1990 .

[20]  Daniel D. Frey,et al.  Engineering design thinking, teaching, and learning , 2006 .

[21]  Jami J. Shah,et al.  Applied Tests of Design Skills—Part 1: Divergent Thinking , 2012 .

[22]  Felix Maringe,et al.  Teaching large classes in an increasingly internationalising higher education environment: pedagogical, quality and equity issues , 2014 .

[23]  Jami J. Shah,et al.  Applied Tests of Design Skills—Part III: Abstract Reasoning , 2014 .

[24]  Richard M. Felder,et al.  The Comprehensive Assessment of Team Member Effectiveness: Development of a Behaviorally Anchored Rating Scale for Self and Peer Evaluation , 2012 .

[25]  Robert J. Anderson,et al.  Representations and Requirements: The Value of Ethnography in System Design , 1994, Hum. Comput. Interact..

[26]  M. Scardamalia,et al.  The CSILE project: Trying to bring the classroom into World 3. , 1994 .

[27]  John S. Gero,et al.  Understanding conceptual electronic design using protocol analysis , 1998 .

[28]  Robin Adams,et al.  What Does it Mean to Design? A Qualitative Investigation of Design Professionals' Experiences , 2012 .

[29]  Hee-Sun Lee,et al.  Measuring Engineering Design Self‐Efficacy , 2010 .

[30]  Joseph B. Cuseo.  The Empirical Case against Large Class Size: Adverse Effects on the Teaching, Learning, and Retention of First-Year Students. , 2007 .