Triangulation of Indicators of Successful Student Design Teams

This paper reports on research conducted on design teams at UC Berkeley over several years at the undergraduate and graduate levels. The paper provides a triangulation of indicators of successful design teams drawn from different research methods. The research sources include questionnaires, team documents, email communication, individual design journals, faculty evaluations, and ratings from external design judges. Computational linguistic algorithms are used to analyze the text documents with a focus on latent semantic analysis and semantic coherence. Sketches are analyzed using a comprehensive list of metrics, including Shah's `variety' measure for quantifying the breadth of the solution space explored during the generation process. A synthesis of the results provides interesting and counterintuitive indicators for predicting the success of student design teams. This analysis, in turn, provides insight into learning how the student design teams negotiate and learn the design process and can assist educators in improving the teaching of design.

[1]  David Craig,et al.  The importance of drawing in the mechanical design process , 1990, Comput. Graph..

[2]  Robert G. Cooper,et al.  Stage-gate systems: A new tool for managing new products , 1990 .

[3]  G. Goldschmidt.  The dialectics of sketching , 1991 .

[4]  Thomas A. Angelo,et al.  Classroom Assessment Techniques: A Handbook for College Teachers. Second Edition. , 1993 .

[5]  Thomas A. Angelo,et al.  Classroom assessment techniques : a handbook for collegeteachers / Thomas A. Angelo, K. Patricia Cross , 1993 .

[6]  Louis L. Bucciarelli,et al.  Designing Engineers , 1994 .

[7]  Karl T. Ulrich,et al.  Product Design and Development , 1995 .

[8]  R. Cooper,et al.  Benchmarking the Firm's Critical Success Factors in New Product Development , 1995 .

[9]  J. Stoker,et al.  Advances in Interdisciplinary Studies of Work Teams , 1997 .

[10]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[11]  Larry Leifer,et al.  Design-Team Performance: Metrics and the Impact of Technology , 1998 .

[12]  Cynthia J. Atman,et al.  Using multiple methods to evaluate a freshmen design course , 2000, 30th Annual Frontiers in Education Conference. Building on A Century of Progress in Engineering Education. Conference Proceedings (IEEE Cat. No.00CH37135).

[13]  Cynthia J. Atman,et al.  Characterizing Engineering Student Design Processes An Illustration Of Iteration , 2000 .

[14]  Shuang Song,et al.  IDENTIFYING SHARED UNDERSTANDING IN DESIGN USING DOCUMENT ANALYSIS , 2001 .

[15]  Clive L. Dym,et al.  Social Dimensions of Engineering Design: Observations from Mudd Design Workshop III (An Educational Brief) , 2001 .

[16]  R. Felder,et al.  The Effects of Personality Type on Engineering Student Performance and Attitudes , 2002 .

[17]  Alice M. Agogino,et al.  Towards Computational Tools for Supporting the Reflective Team , 2002, AID.

[18]  Craig D. Lewis,et al.  Comparing Design Team Self Reports With Actual Performance: Cross Validating Assessment Instruments , 2002 .

[19]  L. Schmidt,et al.  Exploration Of Collective Efficacy Beliefs In Student Project Teams: Implications For Student And Team Outcomes , 2002 .

[20]  Steven M. Smith,et al.  Metrics for measuring ideation effectiveness , 2003 .

[21]  Maria C. Yang,et al.  Concept Generation and Sketching: Correlations With Design Outcome , 2003 .

[22]  Alice M. Agogino,et al.  TIME VARIATION OF DESIGN “STORY TELLING” IN ENGINEERING DESIGN TEAMS , 2003 .

[23]  Alice M. Agogino,et al.  Insights on Designers’ Sketching Activities in New Product Design Teams , 2004 .

[24]  Alice M. Agogino,et al.  A Document Analysis Method for Characterizing Design Team Performance , 2004 .

[25]  Daniel D. Frey,et al.  Engineering design thinking, teaching, and learning , 2006 .