Spanning the complexity chasm: A research approach to move from simple to complex engineering systems

Abstract A multistudy approach is presented that allows design thinking of complex systems to be studied by triangulating causal controlled lab findings with coded data from more complex products. A case study illustration of this approach is provided. During the conceptual design of engineering systems, designers face many cognitive challenges, including design fixation, errors in their mental models, and the sunk cost effect. These factors need to be mitigated for the generation of effective ideas. Understanding the effects of these challenges in a realistic and complex engineering system is especially difficult due to a variety of factors influencing the results. Studying the design of such systems in a controlled environment is extremely challenging because of the scale and complexity of such systems and the time needed to design the systems. Considering these challenges, a mixed-method approach is presented for studying the design thinking of complex engineering systems. This approach includes a controlled experiment with a simple system and a qualitative cognitive-artifacts study on more complex engineering systems followed by the triangulation of results. The triangulated results provide more generalizable information for complex system design thinking. This method combines the advantages of quantitative and qualitative study methods, making them more powerful while studying complex engineering systems. The proposed method is illustrated further using an illustrative study on the cognitive effects of physical models during the design of engineering systems.

[1]  Nigel Cross,et al.  Creativity in the design process: co-evolution of problem–solution , 2001 .

[2]  David W. Rosen,et al.  Refined metrics for measuring ideation effectiveness , 2009 .

[3]  Jonathan Cagan,et al.  A Study of Design Fixation, Its Mitigation and Perception in Engineering Design Faculty , 2010 .

[4]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[5]  Eric Blanco,et al.  Intermediary Objects as a Means to Foster Co-operation in Engineering Design , 2003, Computer Supported Cooperative Work (CSCW).

[6]  Julie S. Linsey,et al.  Role of Sunk Cost in Engineering Idea Generation: An Experimental Investigation , 2013 .

[7]  Tom Kelley,et al.  Prototyping is the shorthand of innovation , 2010 .

[8]  Daniel A. McAdams,et al.  A Component Taxonomy as a Framework for Computational Design Synthesis , 2009, J. Comput. Inf. Sci. Eng..

[9]  John W. Creswell,et al.  Designing and Conducting Mixed Methods Research , 2006 .

[10]  Barbara G. Tabachnick,et al.  Experimental designs using ANOVA , 2006 .

[11]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[12]  Joshua D. Summers,et al.  Case Study Method for Design Research: A Justification , 2008 .

[13]  Elizabeth G. Creamer,et al.  Using a Mixed Methods Content Analysis to Analyze Mission Statements From Colleges of Engineering , 2013 .

[14]  Julie S. Linsey,et al.  Physical Models in Idea Generation: Hindrance or Help? , 2010 .

[15]  R. Weisberg,et al.  Following the wrong footsteps: fixation effects of pictorial examples in a design problem-solving task. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[16]  Carl Auerbach,et al.  Qualitative Data: An Introduction to Coding and Analysis , 2003 .

[17]  G. Altshuller.  Creativity as an exact science : the theory of the solution of inventive problems , 1984 .

[18]  John S. Gero,et al.  Effects of examples on the results of a design activity , 1992, Knowl. Based Syst..

[19]  Daniel Bonner,et al.  A Mixed Methods Evaluation of a 12-Week Insurance-Sponsored Weight Management Program Incorporating Cognitive-Behavioral Counseling , 2010 .

[20]  Amaresh Chakrabarti,et al.  Identification and application of requirements and their impact on the design process: a protocol study , 2004 .

[21]  Jonathan Fish.  Cognitive Catalysis: Sketches for a Time-lagged Brain , 2004 .

[22]  Rocky Ross,et al.  Mental models , 2004, SIGA.

[23]  Joshua D. Summers,et al.  Computer-aided design versus sketching: An exploratory case study , 2012, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[24]  Julie S. Linsey,et al.  Design Fixation in Physical Modeling: An Investigation on the Role of Sunk Cost , 2011 .

[25]  Robert H. McKim,et al.  Experiences in Visual Thinking , 1972 .

[26]  Nancy J. Nersessian,et al.  Turning Experiments into Objects: The Cognitive Processes Involved in the Design of a Lab‐on‐a‐Chip Device , 2013 .

[27]  Jami J. Shah,et al.  Evaluation of idea generation methods for conceptual design: Effectiveness metrics and design of experiments , 2000 .

[28]  Jonathan Cagan,et al.  The role of timing and analogical similarity in the stimulation of idea generation in design , 2008 .

[29]  Maria C. Yang,et al.  A study of prototypes, design activity, and design outcome , 2005 .

[30]  Irem Y. Tumer,et al.  Empirical Findings about Risk and Risk Mitigating Actions from a Legacy Archive of a Large Design Organization , 2013, CSER.

[31]  Carolyn Conner Seepersad,et al.  The Characteristics of Innovative, Mechanical Products , 2011 .

[32]  Kees Dorst,et al.  Briefing and reframing: A situated practice , 2011 .

[33]  Abbas Tashakkori,et al.  Mixed Methodology: Combining Qualitative and Quantitative Approaches , 1998 .

[34]  Joshua D. Summers,et al.  CASE STUDY METHOD FOR DESIGN RESEARCH , 2008 .

[35]  Bo T. Christensen,et al.  The relationship of analogical distance to analogical function and preinventive structure: the case of engineering design , 2007, Memory & cognition.

[36]  Julie S. Linsey,et al.  Physical Models and Design Thinking: A Study of Functionality, Novelty and Variety of Ideas. , 2012 .

[37]  G. S. Altshuller,et al.  40 principles : TRIZ keys to innovation / by GenrichAltshuller , 2005 .

[38]  John S. Gero,et al.  An approach to the analysis of design protocols , 1998 .

[39]  Yin Yin Wong.  Rough and ready prototypes: lessons from graphic design , 1992, CHI '92.

[40]  Donald A. Schön,et al.  Kinds of seeing and their functions in designing , 1992 .

[41]  H. Arkes,et al.  The Psychology of Sunk Cost , 1985 .

[42]  David Radcliffe,et al.  Nature of Rapid Proof-of-concept Prototyping , 1995 .

[43]  Jami J. Shah,et al.  Empirical Studies of Design Thinking: Past, Present, Future , 2013 .

[44]  David Clark-Carter,et al.  Doing Quantitative Psychological Research: From Design To Report , 1997 .

[45]  Arthur B. Markman,et al.  An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods , 2011 .

[46]  Willett Kempton,et al.  Two Theories of Home Heat Control , 1986, Cogn. Sci..

[47]  A. Tenbrunsel,et al.  Organizational Behavior and Human Decision Processes , 2013 .

[48]  Derek F. Sheldon.  Design Review 2005/2006—The ever increasing maturity of design research papers and case studies , 2006 .

[49]  William Lidwell,et al.  Universal Principles of Design , 2003 .

[50]  Sophoria Nicole Westmoreland.  Design thinking: Cognitive patterns in engineering design documentation , 2012 .

[51]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[52]  Steven M. Smith,et al.  Metrics for measuring ideation effectiveness , 2003 .

[53]  R. Lyman Ott.,et al.  An introduction to statistical methods and data analysis , 1977 .

[54]  Masaki Suwa,et al.  What architects see in their sketches: implications for design tools , 1996, CHI Conference Companion.

[55]  Chris Rogers,et al.  The benefits of model building in teaching engineering design , 2010 .

[56]  Cynthia J. Atman,et al.  Engineering Design Processes: A Comparison of Students and Expert Practitioners , 2007 .

[57]  Dorla A. Evans,et al.  The effect of sunk costs on uncertain decisions in experimental markets , 1987 .

[58]  Brigitte Moench,et al.  Engineering Design A Systematic Approach , 2016 .

[59]  Durward K. Sobek,et al.  The Second Toyota Paradox: How Delaying Decisions Can Make Better Cars Faster , 1995 .

[60]  P. M. Wognum,et al.  Introduction to TIPS: a theory for creative design , 1995, Artif. Intell. Eng..

[61]  Kevin Otto,et al.  Product Design: Techniques in Reverse Engineering and New Product Development , 2000 .

[62]  Julie Linsey,et al.  Understanding physical models in design cognition: A triangulation of qualitative and laboratory studies , 2011, 2011 Frontiers in Education Conference (FIE).

[63]  Marian Petre,et al.  How expert engineering teams use disciplines of innovation , 2004 .

[64]  F. B. Vernadat,et al.  Decisions with Multiple Objectives: Preferences and Value Tradeoffs , 1994 .

[65]  A. Onwuegbuzie,et al.  Mixed Methods Research: A Research Paradigm Whose Time Has Come , 2004 .

[66]  V.K. Viswanathan,et al.  Enhancing student innovation: Physical models in the idea generation process , 2009, 2009 39th IEEE Frontiers in Education Conference.

[67]  Joshua D. Summers,et al.  A Proposed Taxonomy for Physical Prototypes: Structure and Validation , 2008, DAC 2008.

[68]  Robert J. Youmans,et al.  The effects of physical prototyping and group work on the reduction of design fixation , 2011 .

[69]  Lucienne Blessing,et al.  DRM, a Design Research Methodology , 2009 .

[70]  Takashi Kiriyama,et al.  Strategic knowledge acquisition: a case study of learning through prototyping , 1998, Knowl. Based Syst..

[71]  G. Goldschmidt.  To see eye to eye: the role of visual representations in building shared mental models in design teams , 2007 .

[72]  N. Nersessian.  Opening the Black Box: Cognitive Science and History of Science , 1995, Osiris.

[73]  Monika Richter,et al.  Cognition In The Wild , 2016 .

[74]  John S. Gero,et al.  Design and other types of fixation , 1996 .

[75]  Paul R. Carlile,et al.  A Pragmatic View of Knowledge and Boundaries: Boundary Objects in New Product Development , 2002, Organ. Sci..

[76]  Xinwei Deng,et al.  Experimental design , 2012, WIREs Data Mining Knowl. Discov..

[77]  Julie S. Linsey,et al.  Design Fixation and Its Mitigation: A Study on the Role of Expertise , 2013 .

[78]  R. G. Fenton,et al.  A MIXED INTEGER-DISCRETE-CONTINUOUS PROGRAMMING METHOD AND ITS APPLICATION TO ENGINEERING DESIGN OPTIMIZATION , 1991 .

[79]  Jami J. Shah,et al.  EMPIRICAL STUDIES OF DESIGN IDEATION: ALIGNMENT OF DESIGN EXPERIMENTS WITH LAB EXPERIMENTS , 2003 .

[80]  Ann F. McKenna,et al.  Characterizing Design Learning: A Mixed‐Methods Study of Engineering Designers' Use of Language , 2008 .