Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment

This paper argues for a need to develop methods for examining temporal patterns in computer-supported collaborative learning (CSCL) groups. It advances one such quantitative method—Lag-sequential Analysis (LsA)—and instantiates it in a study of problem-solving interactions of collaborative groups in an online, synchronous environment. LsA revealed significant temporal patterns in CSCL group discussions that the commonly used “coding and counting” method could not reveal. More importantly, analysis demonstrated how variation in temporal patterns was significantly related to variation in group performance, thereby bolstering the case for developing and testing temporal methods and measures in CSCL research. Findings are discussed, including issues of reliability, validity, and limitations of the proposed method.

[1]  Liam Rourke,et al.  Validity in quantitative content analysis , 2004 .

[2]  Manu Kapur.  Productive failure in mathematical problem solving , 2010 .

[3]  Daniel L. Schwartz,et al.  A time for telling , 1998 .

[4]  Marlene Scardamalia,et al.  Educational Applications of a Networked Communal Database , 1992, Interact. Learn. Environ..

[5]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[6]  Nikol Rummel,et al.  A new method to assess the quality of collaborative process in CSCL , 2005, CSCL.

[7]  Gerry Stahl,et al.  Group cognition in computer-assisted collaborative learning , 2005, J. Comput. Assist. Learn..

[8]  Manu Kapur.  Productive Failure , 2006, ICLS.

[9]  Daniel D. Suthers,et al.  Technology affordances for intersubjective meaning making: A research agenda for CSCL , 2006, Int. J. Comput. Support. Collab. Learn..

[10]  K. VanLehn,et al.  Why Do Only Some Events Cause Learning During Human Tutoring? , 2003 .

[11]  John B. Black,et al.  Insights into the Emergence of Convergence in Group Discussions , 2006, ICLS.

[12]  David H. Jonassen,et al.  Toward a design theory of problem solving , 2000 .

[13]  M. Chi,et al.  The Nature of Expertise , 1988 .

[14]  Manu Kapur,et al.  Examining the effect of problem type in a synchronous computer-supported collaborative learning (CSCL) environment , 2007 .

[15]  John B. Black,et al.  An Emergentist Account of Collective Cognition in Collaborative Problem Solving , 2006 .

[16]  R. Schmidt,et al.  New Conceptualizations of Practice: Common Principles in Three Paradigms Suggest New Concepts for Training , 1992 .

[17]  Gijsbert Erkens,et al.  Computer Support for Collaborative and Argumentative Writing , 2002 .

[18]  Kai Hakkarainen,et al.  Process-and context-sensitive research on academic knowledge practices: developing CASS-tools and methods , 2007, CSCL.

[19]  Stuart A. Kauffman,et al.  At Home in the Universe , 1995 .

[20]  M. Chi.  Quantifying Qualitative Analyses of Verbal Data: A Practical Guide , 1997 .

[21]  Penelope M. Sanderson,et al.  Exploratory Sequential Data Analysis: Foundations , 1994, Hum. Comput. Interact..

[22]  Manu Kapur,et al.  Productive failure in CSCL groups , 2009, Int. J. Comput. Support. Collab. Learn..

[23]  Jan-Willem Strijbos,et al.  Content analysis: What are they talking about? , 2006, Comput. Educ..

[24]  F. Tschan,et al.  Temporal Matters in Social Psychology: Examining the Role of Time in the Lives of Groups and Individuals , 2003 .

[25]  Marlene Scardamalia,et al.  Computer-Supported Intentional Learning Environments , 1989 .

[26]  Yaneer Bar-Yam,et al.  Dynamics Of Complex Systems , 2019 .

[27]  J. Levin,et al.  Single-case research design and analysis : new directions for psychology and education , 1992 .

[28]  César A. Collazos,et al.  Evaluating Collaborative Learning Processes , 2002, CRIWG.

[29]  Pierre Dillenbourg,et al.  Over-scripting CSCL: The risks of blending collaborative learning with instructional design , 2002 .

[30]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[31]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[32]  Manu Kapur,et al.  Problem solving as a complex, evolutionary activity: a methodological framework for analyzing problem-solving processes in a computer-supported collaborative environment , 2005, CSCL.

[33]  Peter Reimann,et al.  Time is precious: Variable- and event-centred approaches to process analysis in CSCL research , 2009, Int. J. Comput. Support. Collab. Learn..

[34]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[35]  Manu Kapur,et al.  A further study of productive failure in mathematical problem solving: unpacking the design components , 2011 .

[36]  Marshall Scott Poole,et al.  Decision Development in Computer-Assisted Group Decision Making , 1995 .

[37]  Wolfgang Banzhaf,et al.  Advances in Artificial Life , 2003, Lecture Notes in Computer Science.

[38]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[39]  Tammy Schellens,et al.  Fostering knowledge construction in university students through asynchronous discussion groups , 2006, Comput. Educ..

[40]  Daniel D. Suthers,et al.  A framework for eclectic analysis of collaborative interaction , 2007, CSCL.

[41]  Brigid Barron.  When Smart Groups Fail , 2003 .

[42]  M. Scardamalia,et al.  Postmodernism, Knowledge Building, and Elementary Science , 1997, The Elementary School Journal.

[43]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[44]  R Sun,et al.  Proceedings of the 28th Annual Conference of the Cognitive Science Society , 2006 .

[45]  Sasha A. Barab,et al.  Constructing Networks of Action-Relevant Episodes: An In Situ Research Methodology , 2001 .

[46]  Manu Kapur,et al.  Sensitivities to early exchange in synchronous computer-supported collaborative learning (CSCL) groups , 2007, Comput. Educ..

[47]  Roger Bakeman,et al.  Observing Interaction: An Introduction to Sequential Analysis , 1986 .

[48]  Janyce Wiebe,et al.  A machine learning approach to assessing knowledge sharing during collaborative learning activities , 2002, CSCL.

[49]  Rachel A. Lotan,et al.  Can Groups Learn , 2002 .

[50]  Lieven Verschaffel,et al.  Powerful learning environments : unravelling basic components and dimensions , 2003 .

[51]  M. Clifford,et al.  Thoughts on a theory of constructive failure , 1984 .

[52]  James D. Herbsleb,et al.  Characterizing the sequential structure of interactive behaviors through statistical and grammatical techniques , 1994 .

[53]  Jörg Zumbach,et al.  Analyzing and supporting collaboration in cooperative computer-mediated communication , 2005, CSCL.

[54]  Mikhail S. Burtsev,et al.  Measuring the Dynamics of Artificial Evolution , 2003, ECAL.

[55]  John R. Koza,et al.  Hidden Order: How Adaptation Builds Complexity. , 1995, Artificial Life.

[56]  Walter R. Borg,et al.  Educational research: An introduction, 6th ed. , 1996 .

[57]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[58]  James F. Kavanagh,et al.  Speech and language in the laboratory, school, and clinic , 1978 .

[59]  Michael J. Jacobson,et al.  Emergence of learning in computer-supported, large-scale collective dynamics: a research agenda , 2007, CSCL.

[60]  Holly Arrow,et al.  Small Groups as Complex Systems: Formation, Coordination, Development, and Adaptation , 2000 .

[61]  D. Jonassen,et al.  Communication patterns in computer mediated versus face-to-face group problem solving , 2001 .

[62]  C. Adami,et al.  Evolution of Biological Complexity , 2000, Proc. Natl. Acad. Sci. USA.

[63]  Allan Jeong.  A Guide to Analyzing Message–Response Sequences and Group Interaction Patterns in Computer‐mediated Communication , 2005 .