Framing and Tracing Human-Centered Design Teams’ Method Selection: An Examination of Decision-Making Strategies

Designers’ choices of methods are well known to shape project outcomes. However, questions remain about why design teams select particular methods and how teams’ decision-making strategies are influenced by projectand process-based factors. In this mixed-methods study, we analyze novice design teams’ decision-making strategies underlying 297 selections of human-centered design methods over the course of three semester-long project-based engineering design courses. We propose a framework grounded in 100+ factors sourced from new product development literature that classifies design teams’ method selection strategy as either Agent(A), Outcome(O), or Process(P) driven, with eight further subclassifications. Coding method selections with this framework, we uncover three insights about design team method selection. First, we identify fewer outcomes-based selection strategies across all phases and innovation types. Second, we observe a shift in decision-making strategy from user-focused outcomes in earlier phases to product-based outcomes in later phases. Third, we observe that decision-making strategy produces a greater heterogeneity of method selections as compared to the class average as a whole, or project type alone. These findings provide a deeper understanding of designers’ method selection behavior and have implications for effective management of design teams, development of automated design support tools to aid design teams, and curation of design method repositories.

[1]  Jitesh H. Panchal,et al.  Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment , 2017 .

[2]  Alice M. Agogino,et al.  Using TheDesignExchange as a knowledge platform for human-centered design-driven global development , 2017 .

[3]  Kemper Lewis,et al.  Cyber-Empathic Design: A Data Driven Framework for Product Design , 2016, DAC 2016.

[4]  Alice M. Agogino,et al.  Sproutel: How Design Roadmapping Helped Improve Children’s Health & Guide a Growing Company , 2018 .

[5]  Paula L. Rechner,et al.  Experiential Effects of Dialectical Inquiry, Devil's Advocacy and Consensus Approaches to Strategic Decision Making , 1989 .

[6]  N. Cross,et al.  Design Expertise Amongst Student Designers , 1994 .

[7]  Christopher McComb,et al.  Capturing Human Sequence-Learning Abilities in Configuration Design Tasks through Markov Chains , 2017 .

[8]  Alice M. Agogino,et al.  Design Talking: How Design Practitioners Talk About Design Research Methods , 2015 .

[9]  Jessica Menold,et al.  Prototyping Canvas: Design Tool for Planning Purposeful Prototypes , 2019, Proceedings of the Design Society: International Conference on Engineering Design.

[10]  S. Oster,et al.  Modern Competitive Analysis , 1989 .

[11]  Fumihiko Kimura,et al.  Design methodologies: Industrial and educational applications , 2009 .

[12]  J. Kimberly,et al.  Organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. , 1981, Academy of Management journal. Academy of Management.

[13]  Cynthia J. Atman,et al.  Engineering Design Processes: A Comparison of Students and Expert Practitioners , 2007 .

[14]  Alice M. Agogino,et al.  Design Roadmapping: A Framework and Case Study on Planning Development of High-Tech Products in Silicon Valley , 2016 .

[15]  Marian S. Kennedy,et al.  Measuring Undergraduate Students' Engineering Self‐Efficacy: A Validation Study , 2016 .

[16]  Claudia Eckert,et al.  What do we need to say about a design method , 2017 .

[17]  Christopher McComb,et al.  Design Strategy Transfer in Cognitively-Inspired Agents , 2018, Volume 2A: 44th Design Automation Conference.

[18]  David J. Singer,et al.  What Is Set-Based Design? , 2009 .

[19]  Scott R. Klemmer,et al.  The efficacy of prototyping under time constraints , 2009, C&C '09.

[20]  Jonathan Cagan,et al.  Expert representation of design repository space: A comparison to and validation of algorithmic output , 2013 .

[21]  Alice M. Agogino,et al.  Pattern Analysis of IDEO's Human-Centered Design Methods in Developing Regions , 2015 .

[22]  Nara L. Newcomer,et al.  User Centered Design , 2014, Encyclopedia of Database Systems.

[23]  Hee-Sun Lee,et al.  Measuring Engineering Design Self‐Efficacy , 2010 .

[24]  Alice M. Agogino,et al.  Deep Learning for Design in Concept Clustering , 2017 .

[25]  Jung-Joo Lee,et al.  Against Method: The Portability of Method in Human-Centered Design , 2012 .

[26]  Mark Fuge,et al.  User Research Methods for Development Engineering: A Study of Method Usage With IDEO’s HCD Connect , 2014 .

[27]  Christopher McComb,et al.  Learning to design from humans: Imitating human designers through deep learning , 2019, Journal of Mechanical Design.

[28]  John H. Friar,et al.  Factors for success in R&D projects and new product innovation: a contextual framework , 1997 .

[29]  D. Madsen,et al.  Not Dead Yet: The Rise, Fall and Persistence of the BCG Matrix , 2017 .

[30]  Jitesh H. Panchal,et al.  Understanding the Impact of Expertise on Design Outcome: An Approach Based on Concept Inventories and Item Response Theory , 2016 .

[31]  Maria C. Yang,et al.  A study of the role of user-centered design methods in design team projects , 2010, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[32]  Alice M. Agogino,et al.  Communicating actionable user research for human-centered design , 2013, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[33]  Herbert Birkhofer,et al.  Why Methods Don't Work and How to Get them to Work , 2002 .

[34]  Alice M. Agogino,et al.  A Document Analysis Method for Characterizing Design Team Performance , 2004 .

[35]  Alice M. Agogino,et al.  Design Talking: An Ontology of Design Methods to Support a Common Language of Design , 2015 .

[36]  Fabrizio Ceschin,et al.  Evolution of design for sustainability: From product design to design for system innovations and transitions , 2016 .

[37]  Daniel Capaldo Amaral,et al.  Automatic digital mood boards to connect users and designers with kansei engineering , 2019, International Journal of Industrial Ergonomics.

[38]  Brigitte Borja de Mozota,et al.  The Impact of User‐Oriented Design on New Product Development: An Examination of Fundamental Relationships* , 2005 .

[39]  Paul E. Green,et al.  Thirty Years of Conjoint Analysis: Reflections and Prospects , 2001, Interfaces.

[40]  Dennis E. Egan,et al.  Handbook of Human Computer Interaction , 1988 .

[41]  Kathryn Weed Jablokow,et al.  The Trajectory of Psychological Safety in Engineering Teams: A Longitudinal Exploration in Engineering Design Education , 2019 .

[42]  Seda Yilmaz,et al.  SUPPORTING IDEA GENERATION THROUGH FUNCTIONAL DECOMPOSITION: AN ALTERNATIVE FRAMING FOR DESIGN HEURISTICS , 2015 .

[43]  Glen A. Livesay,et al.  The Innovation Canvas in Entrepreneurship Education: Integrating Themes of Design, Value, and Market Success , 2014 .

[44]  P. Pintrich,et al.  Beyond Cold Conceptual Change: The Role of Motivational Beliefs and Classroom Contextual Factors in the Process of Conceptual Change , 1993 .

[45]  Phil Hood,et al.  The Power of the 2 x 2 Matrix: Using 2 x 2 Thinking to Solve Business Problems and Make Better Decisions , 2004 .

[46]  George A. Hazelrigg,et al.  An Axiomatic Framework for Engineering Design , 1999 .

[47]  Belinda López-Mesa,et al.  A study of the use of concept selection methods from inside a company , 2011 .

[48]  A. Osborn Applied imagination : principles and procedures of creative problem-solving , 1957 .

[49]  R. Cooper,et al.  Benchmarking the Firm's Critical Success Factors in New Product Development , 1995 .

[50]  R. Weisberg Creativity and knowledge: A challenge to theories. , 1998 .

[51]  Larry Leifer,et al.  Classifying Student Engineering Design Project Types , 2009 .

[52]  I. C. Wright,et al.  The Utilization of Product Development Methods: A Survey of UK Industry , 1996 .

[53]  Sara L. Beckman,et al.  Design Roadmapping in an Uncertain World: Implementing a Customer-Experience-Focused Strategy , 2018, California Management Review.

[54]  Kosa Goucher-Lambert,et al.  The Role of Idea Fluency and Timing on Highly Innovative Design Concepts , 2020 .

[55]  Mark Fuge,et al.  Machine Learning Algorithms for Recommending Design Methods , 2014 .

[56]  C. Shalley,et al.  The Effects of Personal and Contextual Characteristics on Creativity: Where Should We Go from Here? , 2004 .

[57]  David G. Ullman,et al.  Robust decision-making for engineering design , 2001 .

[58]  Beth Dietz-Uhler,et al.  Collective Choice, Judgment, and Problem Solving , 2008 .

[59]  Bill N. Schilit,et al.  Enabling the Internet of Things , 2015, Computer.

[60]  Fabrizio Ceschin,et al.  Critical factors for implementing and diffusing sustainable product-Service systems: insights from innovation studies and companies' experiences , 2013 .

[61]  Charles Hill,et al.  What do Prototypes Prototype , 1997 .

[62]  Maria C. Yang,et al.  Consensus and single leader decision-making in teams using structured design methods , 2010 .

[63]  Sara L. Beckman,et al.  THE DESIGNEXCHANGE: SUPPORTING THE DESIGN COMMUNITY OF PRACTICE , 2011 .

[64]  Jonathan Cagan,et al.  Adaptive Inspirational Design Stimuli: Using Design Output to Computationally Search for Stimuli That Impact Concept Generation , 2020, Journal of Mechanical Design.

[65]  Pieter Jan Stappers,et al.  DesignX: Complex Sociotechnical Systems , 2016 .

[66]  T. M. Amabile The social psychology of creativity: A componential conceptualization. , 1983 .

[67]  L.J. Leifer,et al.  Engineering design thinking, teaching, and learning , 2005, IEEE Engineering Management Review.

[68]  Paul T. Grogan,et al.  Structured to Succeed? Strategy Dynamics in Engineering Systems Design and Their Effect on Collective Performance , 2020, Journal of Mechanical Design.

[69]  Brent D. Rosso Creativity and Constraints: Exploring the Role of Constraints in the Creative Processes of Research and Development Teams , 2014 .

[70]  Carlye Lauff,et al.  What is a Prototype? What are the Roles of Prototypes in Companies? , 2018 .

[71]  Hee-Sun Lee,et al.  Self-Efficacy , 2019, Encyclopedia of Gerontology and Population Aging.

[72]  Kilian Gericke,et al.  An Exploratory Study of the Discovery and Selection of Design Methods in Practice , 2016 .

[73]  Susan Hanley,et al.  Measure what matters , 2014 .

[74]  Su-Cheng Haw,et al.  On Leveraging the Use of Case Studies to Recommend Design Methods: From the Perspective of Human-Centered Design Methodology , 2018 .

[75]  Marc D. Street Groupthink , 1997 .

[76]  Erin Friess The Sword of Data: Does Human-Centered Design Fulfill Its Rhetorical Responsibility , 2010, Design Issues.

[77]  Farrokh Mistree,et al.  An Inverse, Decision-Based Design Method for Robust Concept Exploration , 2020 .

[78]  C. Shalley,et al.  WHAT LEADERS NEED TO KNOW: A REVIEW OF SOCIAL AND CONTEXTUAL FACTORS THAT CAN FOSTER OR HINDER CREATIVITY , 2004 .

[79]  J. Christopher Jones,et al.  Conference on Design Methods : papers presented at the Conference on Systematic and Intuitive Methods in Engineering, Industrial Design, Architecture and Communications, London, September, 1962 , 1963 .

[80]  Jessica Menold,et al.  Prototype for X (PFX): A holistic framework for structuring prototyping methods to support engineering design , 2017 .

[81]  Jung-Joo Lee,et al.  The True Benefits of Designing Design Methods , 2014 .

[82]  Mesut Akdere,et al.  An analysis of decision-making process in organizations: Implications for quality management and systematic practice , 2011 .

[83]  S. Parayitam,et al.  Strategic Decision Making: The Effects of Cognitive Diversity, Conflict, and Trust on Decision Outcomes , 2007 .

[84]  Christopher McComb,et al.  Mining Process Heuristics From Designer Action Data via Hidden Markov Models , 2017 .

[85]  K. Wallace,et al.  Transferring Design Methods into Practice , 2011 .

[86]  Nathan Crilly,et al.  User requirements for analogical design support tools: Learning from practitioners of bio-inspired design , 2017, Design Studies.

[87]  Jitesh H. Panchal,et al.  Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design , 2018, Journal of Mechanical Design.

[88]  H. Chou,et al.  Team cognition, collective efficacy, and performance in strategic decision-making teams , 2012 .

[89]  Triparna de Vreede,et al.  Team Creativity and Innovation: The Effect of Group Composition, Social Processes, and Cognition , 2012 .

[90]  Joshua D. Summers,et al.  Analyzing Requirement Type Influence on Concept Quality and Quantity During Ideation: An Experimental Study , 2014 .

[91]  Christopher McComb,et al.  Transferring Design Strategies From Human To Computer And Across Design Problems , 2019 .

[92]  Alice M. Agogino,et al.  Using “Why and How” to Tap Into Novice Designers’ Method Selection Mindset , 2018 .

[93]  Petra Badke-Schaub,et al.  METHODS IN PRACTICE – A STUDY ON REQUIREMENTS FOR DEVELOPMENT AND TRANSFER OF DESIGN METHODS , 2008 .

[94]  Alice M. Agogino,et al.  Method Selection in Human-Centered Design Teams: An Examination of Decision-Making Strategies , 2020, Volume 8: 32nd International Conference on Design Theory and Methodology (DTM).