Matching Mechanisms to Situations Through the Wisdom of the Crowd

Designing a system often begins with matching existing solutions to current problems. Specifically, integration mechanisms are mapped onto situations. Novices are not good at this task, and experts are rare. Could crowdsourcing, that is, aggregating the suggestions of individuals working independently, be effective? Two experiments, one with design students in a classroom, and another with participants on the web, demonstrated that the crowd possesses wisdom about how to match mechanisms to situations. Participants also categorized situations, and those who name their categories were better at matching than those who didn’t. The results have pragmatic implications, suggesting it is possible to crowdsource design, and providing new ways of eliciting, testing, and training expertise. More generally, the paper suggests a new model for information system design based on analogical mapping.

[1]  Arthur B. Markman,et al.  Defining structural similarity , 2005 .

[2]  Bryan A. Pendleton,et al.  Power of the Few vs. Wisdom of the Crowd: Wikipedia and the Rise of the Bourgeoisie , 2006 .

[3]  Jeffrey D. Ullman,et al.  Principles of Database and Knowledge-Base Systems, Volume II , 1988, Principles of computer science series.

[4]  Jeffrey D. Ullman,et al.  Principles Of Database And Knowledge-Base Systems , 1979 .

[5]  Paul Thagard,et al.  Analogical Mapping by Constraint Satisfaction , 1989, Cogn. Sci..

[6]  L. R. Novick,et al.  Transferring symbolic representations across nonisomorphic problems. , 1994 .

[7]  Frank Harary,et al.  Graph Theory , 2016 .

[8]  K. A. Ericsson,et al.  Recent advances in expertise research: a commentary on the contributions to the special issue , 2005 .

[9]  H. Gurnee.  Maze Learning in the Collective Situation , 1937 .

[10]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[11]  L. R. Novick.  Representational Transfer in Problem Solving , 1990 .

[12]  Dorothea P. Simon,et al.  Expert and Novice Performance in Solving Physics Problems , 1980, Science.

[13]  B. Tversky,et al.  THE SPATIAL NATURE OF THOUGHT: UNDERSTANDING INFORMATION SYSTEMS DESIGN THROUGH DIAGRAMS Le caractère spatial de la pensée : comprendre la conception des systèmes d'information grâce aux diagrammes , 2009 .

[14]  J. Davitz,et al.  A survey of studies contrasting the quality of group performance and individual performance, 1920-1957. , 1958, Psychological bulletin.

[15]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[16]  James E. Smith,et al.  The architecture of virtual machines , 2005, Computer.

[17]  Robert L. Goldstone,et al.  The transfer of abstract principles governing complex adaptive systems , 2003, Cognitive Psychology.

[18]  Gary Lupyan,et al.  Language is not Just for Talking , 2007, Psychological science.

[19]  N. Dalkey,et al.  An Experimental Application of the Delphi Method to the Use of Experts , 1963 .

[20]  H. Simon,et al.  Skill in Chess , 1988 .

[21]  F. Galton.  Vox Populi , 1907, Nature.

[22]  Linda B. Smith,et al.  Object name Learning Provides On-the-Job Training for Attention , 2002, Psychological science.

[23]  Yifan Hu,et al.  Efficient, High-Quality Force-Directed Graph Drawing , 2006 .

[24]  Dorothea P. Simon,et al.  Individual Differences in Solving Physics Problems (1978) , 1978 .

[25]  Yochai Benkler,et al.  The wealth of networks: how social production transforms markets and freedom , 2006 .

[26]  Luis von Ahn.  Games with a Purpose , 2006, Computer.

[27]  Barbara Tversky,et al.  Event Representation in Language and Cognition: Talking about events , 2010 .

[28]  Ilze Zigurs,et al.  A Theory of Task/Technology Fit and Group Support Systems Effectiveness , 1998, MIS Q..

[29]  K. Holyoak,et al.  Surface and structural similarity in analogical transfer , 1987, Memory & cognition.

[30]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[31]  Michael X Cohen,et al.  A Garbage Can Model of Organizational Choice. , 1972 .

[32]  K. Holyoak,et al.  Schema induction and analogical transfer , 1983, Cognitive Psychology.

[33]  R. Hamming,et al.  You and Your Research , 1986 .

[34]  Jeffrey V. Nickerson,et al.  The Spatial Nature of Thought: Understanding Systems Design Through Diagrams , 2008, ICIS.

[35]  Andrea Back,et al.  Group wisdom support systems: Aggregating the insights of many through information technology , 2008 .

[36]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[37]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[38]  Jock Given,et al.  The wealth of networks: How social production transforms markets and freedom , 2007, Inf. Econ. Policy.

[39]  Kalle Lyytinen,et al.  Large Scale Requirements Analysis as Heterogeneous Engineering , 2002, Scand. J. Inf. Syst..

[40]  K. Holyoak,et al.  Analogical problem solving , 1980, Cognitive Psychology.

[41]  K. Gordon,et al.  Group judgments in the field of lifted weights. , 1924 .

[42]  Mark Klein,et al.  Quantifying the costs and benefits of architectural decisions , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[43]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[44]  Richard S. Bruce,et al.  Group Judgments in the Fields of Lifted Weights and Visual Discrimination , 1935 .

[45]  K. Duncker,et al.  On problem-solving , 1945 .

[46]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[47]  Jeffrey V. Nickerson,et al.  Teaching the integration of information systems technologies , 2006, IEEE Transactions on Education.

[48]  Kalle Lyytinen,et al.  Large-Scale Requirements Analysis Revisited: The need for Understanding the Political Ecology of Requirements Engineering , 2002, Requirements Engineering.

[49]  B. Ross,et al.  Generalizing from the use of earlier examples in problem solving , 1990 .

[50]  Donald A. Schön.  The reflective practitioner : how professionals think in action , 1986 .

[51]  Kevin Crowston,et al.  Organizing Business Knowledge: The MIT Process Handbook , 2003 .