Providing different types of group awareness information to guide collaborative learning

Cognitive group awareness tools are a means to guide collaborative learning activities by providing knowledge-related information to the learners. While positive effects of such tools are firmly established, there is no consistency with regard to the awareness information used and a wide range of target concepts exist. However, attempts to compare and integrate the effects of different types of group awareness information are rare. To reduce this gap, our study aims to compare metacognitive and cognitive group awareness information, combining CSCL research and research on metacognition. In our experimental study, 260 university students discussed assumptions on blood-sugar regulation and diabetes mellitus in dyads. We tested the effects of providing cognitive group awareness information on the learners’ assumptions (factor 1) and metacognitive group awareness information on their confidence (factor 2) on individual metacognitive and cognitive outcome measures and on the learners’ regulation of the collaborative process, i.e., the selection of discussion topics based on confidence in knowledge (confidence-based regulation) and based on agreement regarding assumptions (conflict-based regulation). We found that visualizing information strongly impacts joint regulation and that learners seem to integrate the information provided to steer their learning. However, while the learners gained knowledge and confidence during collaboration, providing group awareness information did not have the expected impact on learning outcomes. Reasons and implications of these results in light of previous research on metacognition and group awareness are discussed.

[1]  Pierre Dillenbourg,et al.  Knowing What the Peer Knows: The Differential Effect of Knowledge Awareness on Collaborative Learning Performance of Asymmetric Pairs , 2008, EC-TEL.

[2]  Daniel Bodemer,et al.  Fostering audience design of computer-mediated knowledge communication by knowledge mirroring , 2007, CSCL.

[3]  Daniel Bodemer,et al.  Supporting controversial CSCL discussions with augmented group awareness tools , 2007, Int. J. Comput. Support. Collab. Learn..

[4]  B. Rosenshine,et al.  Teaching Students to Generate Questions: A Review of the Intervention Studies , 1996 .

[5]  Daniel Bodemer,et al.  Knowledge awareness in CSCL: A psychological perspective , 2009, Comput. Hum. Behav..

[6]  Daniel L. Dinsmore,et al.  Focusing the Conceptual Lens on Metacognition, Self-regulation, and Self-regulated Learning , 2008 .

[7]  S. Järvelä,et al.  Socially Shared Regulation of Learning: A Review , 2015 .

[8]  Ruth H. Maki,et al.  Test predictions over text material. , 1998 .

[9]  Mark C. Fox,et al.  When asking the question changes the ultimate answer: Metamemory judgments change memory. , 2016, Journal of experimental psychology. General.

[10]  S. Järvelä,et al.  New Frontiers: Regulating Learning in CSCL , 2013 .

[11]  Pierre Dillenbourg,et al.  Facilitating peer knowledge modeling: Effects of a knowledge awareness tool on collaborative learning outcomes and processes , 2011, Comput. Hum. Behav..

[12]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[13]  Mariel Miller,et al.  Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL , 2015, Comput. Hum. Behav..

[14]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[15]  Rand R Wilcox,et al.  Robust statistical methods: A primer for clinical psychology and experimental psychopathology researchers. , 2017, Behaviour research and therapy.

[16]  Pierre Dillenbourg,et al.  The mechanics of CSCL macro scripts , 2008, Int. J. Comput. Support. Collab. Learn..

[17]  Daniel Bodemer,et al.  What interdependence can tell us about collaborative learning: a statistical and psychological perspective , 2018, Res. Pract. Technol. Enhanc. Learn..

[18]  Daniel Bodemer,et al.  Prompting and visualising monitoring outcomes: Guiding self-regulatory processes with confidence judgments , 2017 .

[19]  Nancy Lowry,et al.  Effects of Controversy on Epistemic Curiosity, Achievement, and Attitudes , 1981 .

[20]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[21]  Daniel D. Suthers,et al.  An Experimental Study of the Effects of Representational Guidance on Collaborative Learning Processes , 2003 .

[22]  W. Crano,et al.  Attitudes and persuasion. , 2006, Annual review of psychology.

[23]  H. H. Clark,et al.  Audience Design in Meaning and Reference , 1982 .

[24]  Friedrich W. Hesse,et al.  Discussion: Being Told to Do Something or Just Being Aware of Something? An Alternative Approach to Scripting in CSCL , 2007 .

[25]  Alejandra Martínez-Monés,et al.  From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning , 2005, Int. J. Artif. Intell. Educ..

[26]  David W. Johnson,et al.  Energizing Learning: The Instructional Power of Conflict , 2009 .

[27]  Jeroen Janssen,et al.  Multilevel analysis in CSCL research , 2011 .

[28]  James Ramirez,et al.  Good probabilistic forecasters: The ‘consumer's’ perspective , 1996 .

[29]  Raymond S. Nickerson,et al.  How We Know—and Sometimes Misjudge—What Others Know: Imputing One's Own Knowledge to Others , 1999 .

[30]  A. King.  Scripting Collaborative Learning Processes: A Cognitive Perspective , 2007 .

[31]  F. Fischer,et al.  Scaffolding and Scripting (Computer-Supported) Collaborative Learning , 2018 .

[32]  A Koriat,et al.  The combined contributions of the cue-familiarity and accessibility heuristics to feelings of knowing. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[33]  Jürgen Buder,et al.  Group awareness tools for learning: Current and future directions , 2011, Comput. Hum. Behav..

[34]  John Dunlosky,et al.  Toward a general model of self-regulated study: An analysis of selection of items for study and self-paced study time. , 1999 .

[35]  R. W. Kulhavy,et al.  Feedback in written instruction: The place of response certitude , 1989 .

[36]  T. O. Nelson,et al.  Allocation of self-paced study time and the "labor-in-vain effect". , 1988, Journal of experimental psychology. Learning, memory, and cognition.

[37]  Daniel M. McNeish,et al.  Psychological Methods Thanks Coefficient Alpha , We ’ ll Take It From Here , 2022 .

[38]  Sanna Järvelä,et al.  Enhancing socially shared regulation in collaborative learning groups: designing for CSCL regulation tools , 2014, Educational Technology Research and Development.

[39]  Jeroen Janssen,et al.  Visualization of agreement and discussion processes during computer-supported collaborative learning , 2007, Comput. Hum. Behav..

[40]  Slava Kalyuga,et al.  Effects of Learner Prior Knowledge and Working Memory Limitations on Multimedia Learning , 2013 .

[41]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[42]  Gijsbert Erkens,et al.  Coordination processes in computer supported collaborative writing , 2005, Comput. Hum. Behav..

[43]  Friedrich W. Hesse,et al.  Providing Group Knowledge Awareness in Computer-Supported Collaborative Learning: Insights into Learning Mechanisms , 2009, Res. Pract. Technol. Enhanc. Learn..

[44]  Gregory Schraw,et al.  Measure for Measure: Calibrating Ten Commonly Used Calibration Scores. , 2013 .

[45]  Dieudonné Leclercq,et al.  Confidence marking: Its use in testing , 1982 .

[46]  Lisa K. Fazio,et al.  Surprising feedback improves later memory , 2009, Psychonomic bulletin & review.

[47]  Dieudonné Leclercq,et al.  Validity, Reliability, and Acuity of Self-Assessment in Educational Testing , 1993 .

[48]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[49]  Friedrich W. Hesse,et al.  Fostering sharing of unshared knowledge by having access to the collaborators' meta-knowledge structures , 2011, Comput. Hum. Behav..

[50]  Laurence Tamatea,et al.  Bourdieu and programming classes for the disadvantaged: a review of current practice as reported online—implications for non-formal coding classes in Bali , 2018, Res. Pract. Technol. Enhanc. Learn..

[51]  F. Fischer,et al.  Toward a Script Theory of Guidance in Computer-Supported Collaborative Learning , 2013, Educational psychologist.

[52]  F. Fischer,et al.  Epistemic and social scripts in computer–supported collaborative learning , 2005 .

[53]  Pierre Dillenbourg,et al.  Collaboration in a multi-user game: impacts of an awareness tool on mutual modeling , 2006, Multimedia Tools and Applications.

[54]  Martin Valcke,et al.  Cognitive load: updating the theory? , 2002 .

[55]  Daniel Bodemer,et al.  Group awareness in CSCL environments , 2011, Comput. Hum. Behav..

[56]  S. Järvelä,et al.  Metacognition in joint discussions: an analysis of the patterns of interaction and the metacognitive content of the networked discussions in mathematics , 2006 .

[57]  Nate Kornell,et al.  Study efficacy and the region of proximal learning framework. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[58]  Kristopher J Preacher,et al.  Statistical mediation analysis with a multicategorical independent variable. , 2014, The British journal of mathematical and statistical psychology.

[59]  J. Stutzman,et al.  The effects of controversy, concurrence seeking, and individualistic learning on achievement and attitude change , 1985 .

[60]  Douglas J. Hacker,et al.  Handbook of Metacognition in Education , 2009 .

[61]  John Dunlosky,et al.  Agenda-based regulation of study-time allocation: when agendas override item-based monitoring. , 2009, Journal of experimental psychology. General.

[62]  Jeroen Janssen,et al.  Group awareness tools: It's what you do with it that matters , 2011, Comput. Hum. Behav..

[63]  Jeroen Janssen,et al.  Group awareness tools for computer-supported collaborative learning , 2018 .

[64]  Hendrik Drachsler,et al.  The pulse of learning analytics understandings and expectations from the stakeholders , 2012, LAK.

[65]  Ton de Jong,et al.  Interaction between tool and talk: how instruction and tools support consensus building in collaborative inquiry-learning environments , 2009, J. Comput. Assist. Learn..

[66]  M. Usher,et al.  Dynamics of metacognitive judgments: pre- and postretrieval mechanisms. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[67]  J. Janssen,et al.  Coordinated Computer-Supported Collaborative Learning: Awareness and Awareness Tools , 2013 .

[68]  Sanna Järvelä,et al.  Socially shared regulation of learning in CSCL: understanding and prompting individual- and group-level shared regulatory activities , 2016, Int. J. Comput. Support. Collab. Learn..

[69]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[70]  Elizabeth Ligon Bjork,et al.  Judgments of learning as memory modifiers. , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[71]  Daniel Bodemer,et al.  Interaction of guidance types and the Need for Cognitive Closure in wiki-based learning , 2018, PeerJ.

[72]  Gijsbert Erkens,et al.  Mediating team effectiveness in the context of collaborative learning: The importance of team and task awareness , 2011, Comput. Hum. Behav..

[73]  J. Metcalfe,et al.  Errors committed with high confidence are hypercorrected. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[74]  Frank Konietschke,et al.  Resampling-Based Analysis of Multivariate Data and Repeated Measures Designs with the R Package MANOVA.RM , 2018, R J..

[75]  J. Sweller.  COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[76]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[77]  J. Metcalfe,et al.  People's hypercorrection of high-confidence errors: did they know it all along? , 2011, Journal of experimental psychology. Learning, memory, and cognition.

[78]  Emma Mercier,et al.  Multi-touch tables and the relationship with collaborative classroom pedagogies: A synthetic review , 2011, Int. J. Comput. Support. Collab. Learn..

[79]  R. Wilcox.  Introduction to Robust Estimation and Hypothesis Testing , 1997 .

[80]  K. Thiede.  The importance of monitoring and self-regulation during multitrial learning , 1999, Psychonomic bulletin & review.

[81]  Jae-Sool Kwon,et al.  Development of an instrument for measuring cognitive conflict in secondary‐level science classes , 2003 .

[82]  Karsten Stegmann,et al.  Facilitating argumentative knowledge construction with computer-supported collaboration scripts , 2007, Int. J. Comput. Support. Collab. Learn..

[83]  J. Metcalfe,et al.  Evidence that judgments of learning are causally related to study choice , 2008, Psychonomic bulletin & review.

[84]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[85]  R. Maki.  Predicting performance on text: Delayed versus immediate predictions and tests , 1998, Memory & cognition.

[86]  John Sweller,et al.  From Cognitive Load Theory to Collaborative Cognitive Load Theory , 2018, International Journal of Computer-Supported Collaborative Learning.

[87]  Objective Assessment of Subjectivity : Degrees of certainty and Partial Knowledge , 2004 .

[88]  Eric R. Stone,et al.  Intuitive evaluation of likelihood judgment producers: evidence for a confidence heuristic , 2004 .

[89]  Raymond W. Kulhavy,et al.  Written Feedback: Response Certitude and Durability. , 1990 .

[90]  Darwin P. Hunt,et al.  The concept of knowledge and how to measure it , 2003 .

[91]  A. Hayes.  Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach , 2013 .

[92]  A. Hayes,et al.  Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation , 2007, Behavior research methods.

[93]  P. Dillenbourg,et al.  Elaborating New Arguments Through a CSCL Script , 2003 .

[94]  Friedrich W. Hesse,et al.  How digital concept maps about the collaborators’ knowledge and information influence computer-supported collaborative problem solving , 2010, Int. J. Comput. Support. Collab. Learn..

[95]  F. Fischer,et al.  A framework to analyze argumentative knowledge construction in computer-supported collaborative learning , 2006, Comput. Educ..

[96]  W. Doise,et al.  Social interaction and the development of cognitive operations , 1975 .

[97]  A. Efklides,et al.  Feeling of difficulty: An aspect of monitoring that influences control , 1999 .

[98]  Lisa K. Son,et al.  Metacognitive and control strategies in study-time allocation. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[99]  David W. Johnson,et al.  An Educational Psychology Success Story: Social Interdependence Theory and Cooperative Learning , 2009 .

[100]  Daniel Bodemer,et al.  Tacit guidance for collaborative multimedia learning , 2011, Comput. Hum. Behav..

[101]  Friedrich W. Hesse,et al.  Guiding knowledge communication in CSCL via group knowledge awareness , 2011, Comput. Hum. Behav..

[102]  Céline Darnon,et al.  Conflict Elaboration and Cognitive Outcomes , 2004 .

[103]  Elizabeth R. Tenney,et al.  Accuracy, confidence, and calibration: how young children and adults assess credibility. , 2011, Developmental psychology.

[104]  Jae-Sool Kwon,et al.  What Do We Know about Students' Cognitive Conflict in Science Classroom: A Theoretical Model of Cognitive Conflict Process. , 2001 .

[105]  Aaltje Hendrikje Gijlers.  Confrontation and co-construction : exploring and supporting collaborative scientic discovery learning with computer simulations , 2005 .

[106]  John Dunlosky,et al.  Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention , 2012 .

[107]  Pierre Dillenbourg,et al.  The Evolution of Research on Computer-Supported Collaborative Learning , 2009 .

[108]  J. Dunlosky,et al.  Updating knowledge about encoding strategies: a componential analysis of learning about strategy effectiveness from task experience. , 2000, Psychology and aging.

[109]  David P Mackinnon,et al.  Explanation of Two Anomalous Results in Statistical Mediation Analysis , 2012, Multivariate behavioral research.

[110]  Daniel D. Suthers,et al.  Towards a Systematic Study of Representational Guidance for Collaborative Learing Discourse , 2001, J. Univers. Comput. Sci..

[111]  Heinz Ulrich Hoppe,et al.  Improving collaborative learning in the classroom: Text mining based grouping and representing , 2016, Int. J. Comput. Support. Collab. Learn..

[112]  P. Kirschner,et al.  Do Learners Really Know Best? Urban Legends in Education , 2013 .

[113]  A. Koriat,et al.  Views That Are Shared With Others Are Expressed With Greater Confidence and Greater Fluency Independent of Any Social Influence , 2016, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[114]  A. King.  Facilitating Elaborative Learning Through Guided Student-Generated Questioning , 1992 .

[115]  Scott H. Fraundorf,et al.  Conflict and metacognitive control: the mismatch-monitoring hypothesis of how others' knowledge states affect recall , 2016, Memory.

[116]  Asher Koriat,et al.  The intricate relationships between monitoring and control in metacognition: lessons for the cause-and-effect relation between subjective experience and behavior. , 2006, Journal of experimental psychology. General.

[117]  Karsten Stegmann,et al.  Collaborative argumentation and cognitive elaboration in a computer-supported collaborative learning environment , 2012 .

[118]  Daniel L. Dinsmore,et al.  What are confidence judgments made of? Students' explanations for their confidence ratings and what that means for calibration , 2013 .

[119]  John Dunlosky,et al.  Utilization of Metacognitive Judgments in the Allocation of Study During Multitrial Learning , 1994 .

[120]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[121]  Raymond W. Kulhavy,et al.  Predicting feedback effects from response-certitude estimates , 1992 .

[122]  Asher Koriat,et al.  The relationships between monitoring, regulation and performance , 2012 .

[123]  M. Vauras,et al.  Socially shared metacognition of dyads of pupils in collaborative mathematical problem-solving processes , 2011 .

[124]  Daniel Bodemer,et al.  Information Cueing in Collaborative Multimedia Learning , 2013, CSCL.

[125]  Pierre Dillenbourg,et al.  Interactive tabletops in education , 2011, Int. J. Comput. Support. Collab. Learn..

[126]  Baruch B. Schwarz,et al.  Visions of CSCL: eight provocations for the future of the field , 2017, Int. J. Comput. Support. Collab. Learn..

[127]  M. Miller,et al.  Leveraging CSCL technology to support and research shared task perceptions in socially shared regulation of learning , 2015 .

[128]  Gregory Schraw.  A conceptual analysis of five measures of metacognitive monitoring , 2009 .

[129]  Christopher Hertzog,et al.  Metacognitive influences on study time allocation in an associative recognition task: An analysis of adult age differences. , 2009, Psychology and aging.

[130]  J. Metcalfe,et al.  A Region of Proximal Learning Model of Study Time Allocation Journal of Memory and Language , 2005 .

[131]  David J. Therriault,et al.  Accuracy of metacognitive monitoring affects learning of texts. , 2003 .

[132]  L. Resnick,et al.  Social foundations of cognition. , 1993, Annual review of psychology.

[133]  J. Metcalfe,et al.  The correction of errors committed with high confidence , 2006 .

[134]  Paul Dourish,et al.  Awareness and coordination in shared workspaces , 1992, CSCW '92.

[135]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[136]  W. Doise,et al.  Socio-cognitive conflict and structure of individual and collective performances , 1978 .