Empirical Performance Evaluation in Collaborative Aircraft Design Tasks

The overarching goal at the Integrated Design Laboratory (IDL) is to understand the mechanisms of decision making and exchanges among engineers. In this study a toolbox for the assessment of engineering performance in a realistic aircraft design task is presented. It allows for the assessment of participants in different experimental conditions. The degree of task difficulty and the amount and quality of visualization are systematically varied across conditions. Using a graphical user interface the participants' mouse trajectories can be tracked. This data together with performance evaluation of the generated aircraft design can help uncover details about the underlying decision making process. The design and the evaluation of the experimental toolbox are presented. This includes the number, specificity, and ranges of design variables that can be manipulated by a participant. The major difficulty thereby is to find a ``sweet spot'' where the task is just difficult enough, such that participants display a progress in their performance. Too easy or too difficult of a task would lead to flooring or ceiling effects, where most participants will always fail or, respectively, perform perfectly. The decisions about the aircraft design parameters are therefore based on a numerical analysis of the design space. With this analysis nonlinearities and interdependencies of design parameters are revealed. The experimental toolbox will be utilized to measure design performance of individuals and groups. The results are expected to reveal ways to support multidisciplinary collaboration.

[1]  Roger Hadgraft,et al.  A systematic consultation process to define graduate outcomes for engineering disciplines , 2011 .

[2]  Volker Gollnick,et al.  An approach to multi-fidelity in conceptual aircraft design in distributed design environments , 2011, 2011 Aerospace Conference.

[3]  T. B. Ward.  Cognition, Creativity, and Entrepreneurship , 2004 .

[4]  Maria C. Yang,et al.  PROMPT VERSUS PROBLEM: HELPING STUDENTS LEARN TO FRAME PROBLEMS AND THINK CREATIVELY , 2007 .

[5]  John R. Rankin,et al.  Robotic Mission to Mars: Hands-on, minds-on, web-based learning , 2012 .

[6]  Petra Badke-Schaub,et al.  EMOTIONAL ALIGNMENT IN TEAMS: HOW EMOTIONS SUPPORT THE DESIGN PROCESS , 2008 .

[7]  Pier Davide Ciampa,et al.  Multidisciplinary Design Optimization in a Collaborative Distributed Aircraft Design System , 2012 .

[8]  Paul Collopy,et al.  The elephant in the mist: What we don't know about the design, development, test and management of complex systems , 2012 .

[9]  Arne Bachmann,et al.  Advances and Work in Progress in Aerospace Predesign Data Exchange , 2010 .

[10]  Lucienne Blessing,et al.  Understanding the differences between how novice and experienced designers approach design tasks , 2003 .

[11]  T. B. Ward,et al.  Expanding Entrepreneurial Cognition's Toolbox: Potential Contributions from the Field of Cognitive Science , 2004 .

[12]  Arne Bachmann,et al.  An Integrated Laboratory for Collaborative Design in the Air Transportation System , 2012, ISPE CE.

[13]  Jesse Austin-Breneman,et al.  Observations of designer behaviors in complex system design , 2011 .