Advancing the Science of Collaborative Problem Solving

Collaborative problem solving (CPS) has been receiving increasing international attention because much of the complex work in the modern world is performed by teams. However, systematic education and training on CPS is lacking for those entering and participating in the workforce. In 2015, the Programme for International Student Assessment (PISA), a global test of educational progress, documented the low levels of proficiency in CPS. This result not only underscores a significant societal need but also presents an important opportunity for psychological scientists to develop, adopt, and implement theory and empirical research on CPS and to work with educators and policy experts to improve training in CPS. This article offers some directions for psychological science to participate in the growing attention to CPS throughout the world. First, it identifies the existing theoretical frameworks and empirical research that focus on CPS. Second, it provides examples of how recent technologies can automate analyses of CPS processes and assessments so that substantially larger data sets can be analyzed and so students can receive immediate feedback on their CPS performance. Third, it identifies some challenges, debates, and uncertainties in creating an infrastructure for research, education, and training in CPS. CPS education and assessment are expected to improve when supported by larger data sets and theoretical frameworks that are informed by psychological science. This will require interdisciplinary efforts that include expertise in psychological science, education, assessment, intelligent digital technologies, and policy.

[1]  Richard J. Klimoski,et al.  Training Undergraduates to Work in Organizational Teams , 2004 .

[2]  A. Graesser,et al.  Collaborative problem-solving education for the twenty-first-century workforce , 2018, Nature Human Behaviour.

[3]  J. Roschelle.  Learning by Collaborating: Convergent Conceptual Change , 1992 .

[4]  Robert E. Slavin,et al.  Instruction Based on Cooperative Learning , 2010 .

[5]  C. MacCann,et al.  DEVELOPMENT AND VALIDITY EVIDENCE SUPPORTING A TEAMWORK AND COLLABORATION ASSESSMENT FOR HIGH SCHOOL STUDENTS , 2008 .

[6]  Marc D. Street.  Groupthink , 1997 .

[7]  S. Fiore.  Interdisciplinarity as Teamwork , 2008 .

[8]  B. Mullen,et al.  The relation between group cohesiveness and performance: An integration. , 1994 .

[9]  P. Costa,et al.  Revised NEO Personality Inventory (NEO-PI-R) and NEO-Five-Factor Inventory (NEO-FFI) , 1992 .

[10]  Stuart W. Elliott.  Computers and the future of skill demand , 2017 .

[11]  Eduardo Salas,et al.  From Data, to Information, to Knowledge: Measuring Knowledge Building in the Context of Collaborative Cognition , 2018, Macrocognition Metrics and Scenarios.

[12]  Nikol Rummel,et al.  A rating scheme for assessing the quality of computer-supported collaboration processes , 2007, Int. J. Comput. Support. Collab. Learn..

[13]  Michael J. Timms,et al.  Research Article Science Assessments for All: Integrating Science Simulations Into Balanced State Science Assessment Systems , 2012 .

[14]  Filip Lievens,et al.  Adjusting medical school admission: assessing interpersonal skills using situational judgement tests , 2013, Medical education.

[15]  Georgia T. Chao,et al.  The Dynamics of Team Cognition: , 2016 .

[16]  N. Kerr.  Motivation losses in small groups: a social dilemma analysis , 1983 .

[17]  Robert L. Goldstone,et al.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Emergent Processes in Group Behavior , 2022 .

[18]  Antony Stephen Reid Manstead,et al.  Antecedents and Consequences of Group-Based Guilt: The Effects of Ingroup Identification , 2006 .

[19]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[20]  Nancy J. Cooke,et al.  Enhancing the Effectiveness of Team Science , 2015 .

[21]  Eduardo Salas,et al.  Does Team Building Work? , 2009 .

[22]  Linda C. Angell,et al.  Personality Heterogeneity in Teams , 2003 .

[23]  Christopher D. Hundhausen,et al.  Does studio-based instruction work in CS 1?: an empirical comparison with a traditional approach , 2010, SIGCSE.

[24]  Daniel R. Ilgen,et al.  A century of work teams in the Journal of Applied Psychology. , 2017, The Journal of applied psychology.

[25]  K. Williams,et al.  Many Hands Make Light the Work: The Causes and Consequences of Social Loafing , 1979 .

[26]  Arthur C. Graesser,et al.  Conversations with AutoTutor Help Students Learn , 2016, International Journal of Artificial Intelligence in Education.

[27]  Arthur C. Graesser,et al.  Assessing conversation quality, reasoning, and problem solving with computer agents , 2017 .

[28]  Brent Morgan,et al.  Assessment with computer agents that engage in conversational dialogues and trialogues with learners , 2017, Comput. Hum. Behav..

[29]  Robyn M. Gillies,et al.  The Effects of Cooperative Learning on Junior High School Students' Behaviours, Discourse and Learning During a Science-Based Learning Activity , 2008 .

[30]  Michael J. Baker,et al.  Promoting reflective interactions in a CSCL environment , 1997, J. Comput. Assist. Learn..

[31]  Robert J. Mislevy,et al.  The In‐Task Assessment Framework for Behavioral Data , 2016 .

[32]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[33]  Larry K. Michaelsen,et al.  Team-based learning for health professions education : a guide to using small groups for improving learning , 2008 .

[34]  Lindred L. Greer,et al.  THE BLACK-BOX DECIPHERED: A META-ANALYSIS OF TEAM DIVERSITY, CONFLICT, AND TEAM PERFORMANCE. , 2008 .

[35]  Yigal Rosen,et al.  Comparability of Conflict Opportunities in Human-to-Human and Human-to-Agent Online Collaborative Problem Solving , 2014, Technol. Knowl. Learn..

[36]  Esther Care,et al.  Assessment of Twenty-First Century Skills: The Issue of Authenticity , 2018 .

[37]  G R Norman,et al.  Effectiveness of problem‐based learning curricula: theory, practice and paper darts , 2000, Medical education.

[38]  Karl A. Smith,et al.  Constructive Controversy: The Educative Power of Intellectual Conflict , 2000 .

[39]  Gregory K. W. K. Chung,et al.  Issues in the Computer-based Assessment of Collaborative Problem Solving , 2003 .

[40]  D. Wegner.  Transactive Memory: A Contemporary Analysis of the Group Mind , 1987 .

[41]  Claire Scoular,et al.  Measuring Collaborative Problem Solving Using Mathematics-Based Tasks , 2017 .

[42]  F. Fischer,et al.  Socio-Cognitive Scaffolding with Computer-Supported Collaboration Scripts: a Meta-Analysis , 2017 .

[43]  Murray R. Barrick,et al.  Relating member ability and personality to work-team processes and team effectiveness. , 1998 .

[44]  Joe Alper,et al.  Developing a National STEM Workforce Strategy: A Workshop Summary. , 2016 .

[45]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[46]  Stephen M. Fiore,et al.  Toward an Understanding of Macrocognition in Teams: Predicting Processes in Complex Collaborative Contexts , 2010, Hum. Factors.

[47]  Robert L. Goldstone,et al.  Author ' s personal copy Recognizing group cognition Action editor : , 2010 .

[48]  Stephen M. Fiore,et al.  What can Training Researchers Gain from Examination of Methods for Active-Learning (PBL, TBL, and SBL) , 2013 .

[49]  Monika Richter,et al.  Cognition In The Wild , 2016 .

[50]  Arthur C. Graesser,et al.  Building Intelligent Conversational Tutors and Mentors for Team Collaborative Problem Solving: Guidance from the 2015 Program for International Student Assessment , 2018, Research on Managing Groups and Teams.

[51]  E. Salas,et al.  Reflections on shared cognition , 2001 .

[52]  James W. Pennebaker,et al.  Improving teamwork using real-time language feedback , 2013, CHI.

[53]  Arthur C. Graesser,et al.  Challenges of Assessing Collaborative Problem Solving , 2018 .

[54]  Arthur C. Graesser,et al.  Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions , 2018, Behavior Research Methods.

[55]  Patrick Griffin,et al.  The ATC21S Method , 2015 .

[56]  Leslie A. DeChurch,et al.  Information sharing and team performance: a meta-analysis. , 2009, The Journal of applied psychology.

[57]  Joachim Funke,et al.  Complex problem solving: a case for complex cognition? , 2010, Cognitive Processing.

[58]  Paul Christiansen,et al.  The impact of own and others’ alcohol consumption on social contagion following a collaborative memory task , 2018, Memory.

[59]  A. Graesser,et al.  Domain-general problem solving skills and education in the 21st century , 2014 .

[60]  Kay G. Schulze,et al.  An Analysis of Communication and Use of Military Terms in Navy Team Training , 1995 .

[61]  Eduardo Salas,et al.  Complex collaborative problem-solving processes in mission control. , 2014, Aviation, space, and environmental medicine.

[62]  Michelle L Meade,et al.  Both young and older adults discount suggestions from older adults on a social memory test , 2013, Psychonomic Bulletin & Review.

[63]  Dennis J. Devine,et al.  A Meta-Analysis of Cohesion and Performance , 1995 .

[64]  Arthur C. Graesser,et al.  AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back , 2012, TIIS.

[65]  Robert L. Goldstone,et al.  The multiple, interacting levels of cognitive systems (MILCS) perspective on group cognition , 2017 .

[66]  D. Steele-Johnson,et al.  The Effects of Team Training on Team Outcomes: A Meta-Analysis , 2010 .

[67]  Kristen E. DiCerbo,et al.  Skills for Today: What We Know about Teaching and Assessing Collaboration. , 2017 .

[68]  Carolyn Penstein Rosé,et al.  The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions , 2012, Int. J. Comput. Support. Collab. Learn..

[69]  Melanie J. Martin,et al.  Analysis of Semantic Content and Its Relation to Team Neurophysiology during Submarine Crew Training , 2013, HCI.

[70]  Suzanne T Bell,et al.  Deep-level composition variables as predictors of team performance: a meta-analysis. , 2007, The Journal of applied psychology.

[71]  John Sutton,et al.  Building consensus about the past: Schema consistency and convergence in socially shared retrieval-induced forgetting , 2010, Memory.

[72]  James A Grand,et al.  The dynamics of team cognition: A process-oriented theory of knowledge emergence in teams. , 2016, The Journal of applied psychology.

[73]  Arthur C. Graesser,et al.  Collaborative Problem Solving: Considerations for the National Assessment of Educational Progress , 2017 .

[74]  R. Liden,et al.  Social Loafing: A Field Investigation , 2004 .

[75]  Antonija Mitrovic,et al.  Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams , 2007, Int. J. Comput. Support. Collab. Learn..

[76]  W. Doise.  The social development of the intellect , 1984 .

[77]  Robert A. Sottilare,et al.  Designing Adaptive Instruction for Teams: a Meta-Analysis , 2017, International Journal of Artificial Intelligence in Education.

[78]  Arthur C. Graesser,et al.  Collaborative problem solving education for the 21st century workforce , 2018 .

[79]  Clay Spinuzzi,et al.  Context and consciousness: Activity theory and human-computer interaction , 1997 .

[80]  J. Mathieu,et al.  Modeling reciprocal team cohesion-performance relationships, as impacted by shared leadership and members' competence. , 2015, The Journal of applied psychology.

[81]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[82]  Pierre Dillenbourg,et al.  Collaborative Learning: Cognitive and Computational Approaches , 1999 .

[83]  J. Flavell.  Metacognitive aspects of problem solving , 1976 .

[84]  B. B. Morgan,et al.  An Analysis of Team Evolution and Maturation , 1993 .

[85]  Päivi Häkkinen,et al.  Effect of Ca2+, cyclic GMP, and cyclic AMP added to artificial solution perfusing lingual artery on frog gustatory nerve responses , 1982, The Journal of general physiology.

[86]  Claire Scoular,et al.  Teaching Twenty-First Century Skills: Implications at System Levels in Australia , 2018 .

[87]  M. A. Campion,et al.  The Knowledge, Skill, and Ability Requirements for Teamwork: Implications for Human Resource Management , 1994 .

[88]  Arthur C. Graesser,et al.  Assessment of collaborative problem solving. , 2017 .

[89]  James W. Smither,et al.  APPLICANT REACTIONS TO SELECTION PROCEDURES , 2006 .

[90]  P. Griffin,et al.  Assessment of Collaborative Problem Solving in Education Environments , 2016 .

[91]  H L Roediger,et al.  Social contagion of memory , 2001, Psychonomic bulletin & review.

[92]  Eduardo Salas,et al.  Towards an understanding of macrocognition in teams: developing and defining complex collaborative processes and products , 2010 .

[93]  C. D. De Dreu,et al.  Work group diversity and group performance: an integrative model and research agenda. , 2004, The Journal of applied psychology.

[94]  Robert E. Ployhart,et al.  An Evaluation of Generic Teamwork Skills Training with Action Teams: Effects on Cognitive and Skill-Based Outcomes , 2005 .

[95]  J. Pellegrino,et al.  Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century , 2013 .

[96]  Graham Hitch,et al.  Effects of the timing and identity of retrieval cues in individual recall: An attempt to mimic cross-cueing in collaborative recall , 2006, Memory.

[97]  J. Koenig.  Assessing 21st Century Skills: Summary of a Workshop , 2011 .

[98]  Samuel Greiff,et al.  The assessment of collaborative problem solving in PISA 2015: Can computer agents replace humans? , 2020, Comput. Hum. Behav..

[99]  Russell G. Almond,et al.  On the Structure of Educational Assessments, CSE Technical Report. , 2003 .

[100]  Michael C. Dorneich,et al.  Creating a Team Tutor Using GIFT , 2018, International Journal of Artificial Intelligence in Education.

[101]  Uno Fors,et al.  Assessing Collaborative Problem Solving Skills in Technology-Enhanced Learning Environments - The PISA Framework and Modes of Communication , 2017, Int. J. Emerg. Technol. Learn..

[102]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[103]  Patrick F. Reidy.  An Introduction to Latent Semantic Analysis , 2009 .

[104]  Peter W. Foltz,et al.  Automated Communication Analysis of Teams , 2008 .

[105]  Carol Forsyth,et al.  Exploring social and cognitive dimensions of collaborative problem solving in an open online simulation-based task , 2020, Comput. Hum. Behav..

[106]  Friedrich W. Hesse,et al.  Designs for Operationalizing Collaborative Problem Solving for Automated Assessment , 2017 .

[107]  Mengxiao Zhu,et al.  Innovative Assessment of Collaboration , 2017 .

[108]  Ronald F. Piccolo,et al.  A META‐ANALYSIS OF TEAMWORK PROCESSES: TESTS OF A MULTIDIMENSIONAL MODEL AND RELATIONSHIPS WITH TEAM EFFECTIVENESS CRITERIA , 2008 .

[109]  Kevin Crowston,et al.  Coordination Theory: A Ten-Year Retrospective , 2004, Computer Supported Acitivity Coordination.

[110]  Travis J. Wiltshire,et al.  Problem-Solving Phase Transitions During Team Collaboration , 2018, Cogn. Sci..

[111]  Alan R. Dennis,et al.  A Meta-Analysis of Group Side Effects in Electronic Brainstorming: More Heads are Better than One , 2005, Int. J. e Collab..

[112]  A. Graesser,et al.  Language and Discourse Analysis with Coh-Metrix: Applications from Educational Material to Learning Environments at Scale , 2016, J. Learn. Anal..

[113]  N. Nguyen,et al.  Situational Judgment Tests: A Review of Practice and Constructs Assessed , 2001 .

[114]  Samuel Greiff,et al.  The assessment of collaborative problem solving in PISA 2015: An investigation of the validity of the PISA 2015 CPS tasks , 2018, Comput. Educ..

[115]  R. Hastie,et al.  Proper analysis of the accuracy of group judgments , 1997 .

[116]  Lei Liu,et al.  A Tough Nut to Crack: Measuring Collaborative Problem Solving , 2016 .

[117]  Patrick Griffin,et al.  Assessment and teaching of 21st century skills : methods and approach , 2015 .

[118]  D. van Knippenberg,et al.  Work group diversity. , 2007, Annual review of psychology.

[119]  Unité d'analyse,et al.  PISA 2012 FIELD TRIAL PROBLEM SOLVING FRAMEWORK SUBJECT TO POSSIBLE REVISION AFTER THE FIELD , 2012 .

[120]  Suparna Rajaram,et al.  Collaboration Both Hurts and Helps Memory , 2011 .

[121]  Carol Forsyth,et al.  Identifying Profiles of Collaborative Problem Solvers in an Online Electronics Environment , 2018, EDM.

[122]  K. Williams,et al.  Identifiability as a deterrant to social loafing: Two cheering experiments. , 1981 .

[123]  Reid Hastie,et al.  The robust beauty of majority rules in group decisions. , 2005, Psychological review.

[124]  M. Segers,et al.  Effects of problem-based learning: a meta- analysis , 2003 .

[125]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[126]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[127]  Peter W. Foltz,et al.  Automated Team Discourse Annotation and Performance Prediction Using LSA , 2004, NAACL.

[128]  Hsi-An Shih,et al.  Team diversity and team helping behavior: The mediating roles of team cooperation and team cohesion , 2015 .

[129]  Nancy J. Cooke,et al.  On Teams, Teamwork, and Team Performance: Discoveries and Developments , 2008, Hum. Factors.

[130]  Stephen M. Fiore,et al.  Processes in complex team problem-solving: Parsing and defining the theoretical problem space , 2008 .

[131]  Neil Brewer,et al.  Memory conformity for confidently recognized items: The power of social influence on memory reports , 2012 .

[132]  Stephen H. Wagner,et al.  The Relationship between Work-Team Personality Composition and the Job Performance of Teams , 1999 .

[133]  Samuel Greiff,et al.  The Computer-Based Assessment of Complex Problem Solving and How It Is Influenced by Students' Information and Communication Technology Literacy. , 2014 .

[134]  Kipling D. Williams,et al.  PROCESSES Social Loafing: A Meta-Analytic Review and Theoretical Integration , 2022 .

[135]  D. Beal,et al.  Cohesion and performance in groups: a meta-analytic clarification of construct relations. , 2003, The Journal of applied psychology.

[136]  E. Salas,et al.  Productivity loss in brainstorming groups: A meta-analytic integration. , 1991 .

[137]  Nancy J. Cooke,et al.  Macrocognition in Teams: Theories and Methodologies , 2017 .

[138]  Peter Brusilovsky.  Domain Modeling for Personalized Guidance , 2016 .

[139]  Jessica J. Andrews,et al.  Partner Characteristics and Social Contagion: Does Group Composition Matter? , 2014 .

[140]  Patrick Griffin,et al.  Assessment and Teaching of 21st Century Skills , 2011 .

[141]  Dan MacIsaac,et al.  Developing a National STEM Workforce Strategy: A workshop summary , 2016 .

[142]  James C. Lester,et al.  Face-to-Face Interaction with Pedagogical Agents, Twenty Years Later , 2016, International Journal of Artificial Intelligence in Education.

[143]  Anne Maass,et al.  The role of social categorization and perceived source credibility in minority influence , 1988 .

[144]  Joachim Funke,et al.  The nature of problem solving. Using research to inspire 21st century learning , 2017 .

[145]  Patrick Griffin,et al.  Assessment of collaborative problem-solving processes , 2017 .

[146]  Pierre Dillenbourg,et al.  Sharing Solutions: Persistence and Grounding in Multimodal Collaborative Problem Solving , 2006 .

[147]  D. T. Vernon,et al.  Does problem‐based learning work? A meta‐analysis of evaluative research , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[148]  Mary Susan Weldon,et al.  Collective memory: collaborative and individual processes in remembering. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[149]  N. Augustine.  Rising Above The Gathering Storm: Energizing and Employing America for a Brighter Economic Future , 2006 .

[150]  Joan Harman,et al.  A Quantitative Integration of the Military Cohesion Literature , 1999 .

[151]  Samuel Greiff,et al.  The Process of Solving Complex Problems , 2012, J. Probl. Solving.

[152]  J. Vermunt,et al.  Congruence and friction between learning and teaching , 1999 .

[153]  Stephen M Fiore,et al.  The Science of Team Science: A Review of the Empirical Evidence and Research Gaps on Collaboration in Science , 2018, The American psychologist.

[154]  Walter Gropius,et al.  Bauhaus 1919-1928 , 1959 .

[155]  Ute Fischer,et al.  Linguistic correlates of team performance: toward a tool for monitoring team functioning during space missions. , 2007, Aviation, space, and environmental medicine.

[156]  E. Aronson,et al.  The Jigsaw Classroom: Building Cooperation in the Classroom , 1997 .

[157]  Travis J. Wiltshire,et al.  Technology as Teammate: Examining the Role of External Cognition in Support of Team Cognitive Processes , 2016, Front. Psychol..

[158]  William F. McComas,et al.  “21st-Century Skills” , 2014 .

[159]  C. R. Evans,et al.  Group Cohesion and Performance , 1991 .

[160]  Susan R. Fussell,et al.  Conversational Argumentation in Decision Making: Chinese and U.S. Participants in Face-to-Face and Instant-Messaging Interactions , 2007 .

[161]  Ronald Maier,et al.  Macrocognition in Collaboration: Analyzing Processes of Team Knowledge Building with CoPrA , 2013, Group Decision and Negotiation.

[162]  C. Hmelo‐Silver.  Problem-Based Learning: What and How Do Students Learn? , 2004 .

[163]  Friedrich W. Hesse,et al.  A Framework for Teachable Collaborative Problem Solving Skills , 2015 .

[164]  Wim H. Gijselaers,et al.  Connecting problem‐based practices with educational theory , 1996 .

[165]  Vasile Rus,et al.  SEMILAR: The Semantic Similarity Toolkit , 2013, ACL.

[166]  Peter W. Foltz,et al.  Assessing Collaborative Problem Solving Through Automated Technologies , 2014 .

[167]  Friedrich W. Hesse,et al.  Guiding knowledge communication in CSCL via group knowledge awareness , 2011, Comput. Hum. Behav..

[168]  H. H. Clark,et al.  Understanding by addressees and overhearers , 1989, Cognitive Psychology.

[169]  William Hirst,et al.  Silence Is Not Golden , 2007, Psychological science.

[170]  Jessica J. Andrews,et al.  Benefits, Costs, and Challenges of Collaboration for Learning and Memory , 2015 .