Using crowdsourcing to provide analogies for designer ideation in a cognitive study

Analogical reasoning is a prominent method for human creative design. The design research community has invested significant effort into understanding the process of design by analogy, including the impact of different types of analogies on design thinking and solution characteristics. Yet, generating those analogies is a challenge. The present work investigates whether it is possible to obtain useful analogies from individuals with no domain knowledge. To do this, individuals in a crowd workforce were asked to provide solutions for design problems previously explored in the literature. A text mining approach was used to extract commonly used words from these responses, which then served as analogies for problem solvers with design expertise. Finally, 111 participants were recruited for a cognitive study in which they were asked to solve four design problems using some subset of crowd-sourced analogical inspiration. Results indicate that it is feasible to gather impactful analogies from a crowd workforce. The usefulness of analogies at different analogical distances is highly dependent on the problem itself, highlighting the utility of obtaining analogies using the crowd.

[1]  Michael I. Norton,et al.  Local Motors: Designed by the Crowd, Built by the Customer , 2011 .

[2]  Arthur B. Markman,et al.  Modality and representation in analogy , 2008, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[3]  Arthur B. Markman,et al.  Wordtrees: A Method For Design By Analogy , 2008 .

[4]  Jonathan Cagan,et al.  The role of timing and analogical similarity in the stimulation of idea generation in design , 2008 .

[5]  Brian P. Bailey,et al.  Exploring the Utility of Bayesian Truth Serum for Assessing Design Knowledge , 2014, Hum. Comput. Interact..

[6]  Arthur B. Markman,et al.  Design by Analogy: A Study of the WordTree Method for Problem Re-Representation , 2012 .

[7]  Gabriela Goldschmidt,et al.  Expertise and the use of visual analogy: implications for design education , 1999 .

[8]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[9]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[10]  Christian D. Schunn,et al.  Do the best design ideas (really) come from conceptually distant sources of inspiration , 2015 .

[11]  Maria C. Yang,et al.  Fundamental studies in Design-by-Analogy: A focus on domain-knowledge experts and applications to transactional design problems , 2014 .

[12]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[13]  Gabriela Goldschmidt,et al.  Variances in the impact of visual stimuli on design problem solving performance , 2006 .

[14]  Cornelius Herstatt,et al.  Developing Innovations Based on Analogies: Experience from Design and Engineering Consultants , 2010 .

[15]  Jonathan Cagan,et al.  On the benefits and pitfalls of analogies for innovative design : Ideation performance based on analogical distance, commonness, and modality of examples , 2011 .

[16]  Scarlett R. Miller,et al.  The Impact of Example Modality and Physical Interactions on Design Creativity , 2014 .

[17]  Julie S. Linsey,et al.  Overcoming Cognitive Challenges in Bioinspired Design and Analogy , 2014 .

[18]  Mandy J. Maguire,et al.  An fMRI investigation of cognitive stages in reasoning by analogy , 2010, Brain Research.

[19]  Ashok K. Goel,et al.  A content account of creative analogies in biologically inspired design , 2010, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[20]  Petra Badke-Schaub,et al.  The Influence of Different Pictorial Representations during Idea Generation. , 2011 .

[21]  Yi Ren,et al.  When Crowdsourcing Fails: A Study of Expertise on Crowdsourced Design Evaluation , 2015 .

[22]  John S. Gero,et al.  Fixation Effects: Do They Exist in Design Problem Solving? , 1993 .

[23]  D. Cicchetti.  Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. , 1994 .

[24]  Julie S. Linsey,et al.  Design Fixation and Its Mitigation: A Study on the Role of Expertise , 2013 .

[25]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[26]  David Wallace,et al.  Assessing the Quality of Ideas From Prolific, Early-Stage Product Ideation , 2010 .