Towards the real-time evaluation of collaborative activities: Integration of an automatic rater of collaboration quality in the classroom from the teacher’s perspective

This paper presents the integration of a real time evaluation method of collaboration quality in a monitoring application that supports teachers in class orchestration. The method is implemented as an automatic rater of collaboration quality and studied in a real time scenario of use. We argue that automatic and semi-automatic methods which provide assessments of collaboration quality or detect disorders and anomalies in collaborative activities should be further used to assist class monitoring. To that end, we study the routine of teachers when they monitor computer-supported collaborative activities with respect to the size of the classroom and the support of an automatic rater. We focus on the way the rater affects their task and how it can be further used to empower the practice of teachers.

[1]  Daniel D. Suthers,et al.  Computer-supported collaborative learning: An historical perspective , 2006 .

[2]  Julita Vassileva,et al.  Collaboration and Technology , 2014, Lecture Notes in Computer Science.

[3]  F. Fischer,et al.  A framework to analyze argumentative knowledge construction in computer-supported collaborative learning , 2006, Comput. Educ..

[4]  Daniel D. Suthers,et al.  An approach for coaching collaboration based on difference recognition and participation tracking , 2007 .

[5]  C. Hmelo‐Silver,et al.  Analyzing Interactions in CSCL , 2011 .

[6]  Alejandra Martínez-Monés,et al.  An Interaction-Aware Design Process for the Integration of Interaction Analysis into Mainstream CSCL Practices , 2011 .

[7]  Nikolaos Avouris,et al.  Modelling interaction during small-group synchronous problem-solving activities: The Synergo approach . , 2004 .

[8]  Georgios Kahrimanis,et al.  Validating Empirically a Rating Approach for Quantifying the Quality of Collaboration , 2012, Intelligent Adaptation and Personalization Techniques in Computer-Supported Collaborative Learning.

[9]  Toni Giorgino,et al.  Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package , 2009 .

[10]  Georgios Kahrimanis,et al.  Interaction Analysis as a Tool for Supporting Collaboration: An Overview , 2011, Technology-Enhanced Systems and Tools for Collaborative Learning Scaffolding.

[11]  María Jesús Rodríguez-Triana,et al.  Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations , 2015, Br. J. Educ. Technol..

[12]  Nikol Rummel,et al.  A new method to assess the quality of collaborative process in CSCL , 2005, CSCL.

[13]  Ralph Arnote,et al.  Hong Kong (China) , 1996, OECD/G20 Base Erosion and Profit Shifting Project.

[14]  Nikolaos M. Avouris,et al.  Support for the teacher in technology-enhanced collaborative classroom , 2012, Education and Information Technologies.

[15]  H. U. Hoppe,et al.  The Role of Technology in CSCL , 2007 .

[16]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[17]  Anouschka van Leeuwen,et al.  Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL , 2014, Comput. Educ..

[18]  Nikolaos M. Avouris,et al.  It's All about Time: Towards the Real-Time Evaluation of Collaborative Activities , 2014, 2014 IEEE 14th International Conference on Advanced Learning Technologies.

[19]  Alejandra Martínez-Monés,et al.  From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning , 2005, Int. J. Artif. Intell. Educ..

[20]  Fatos Xhafa,et al.  Technology-Enhanced Systems and Tools for Collaborative Learning Scaffolding , 2011, Technology-Enhanced Systems and Tools for Collaborative Learning Scaffolding.

[21]  Nikol Rummel,et al.  Assessing Collaboration Quality in Synchronous CSCL Problem-Solving Activities: Adaptation and Empirical Evaluation of a Rating Scheme , 2009, EC-TEL.

[22]  Heinz Ulrich Hoppe,et al.  Effects of awareness support on moderating multiple parallel E-discussions , 2009, CSCL.

[23]  Nikolaos M. Avouris,et al.  Time Series Analysis of Collaborative Activities , 2012, CRIWG.

[24]  Daniel D. Suthers,et al.  A framework for conceptualizing, representing, and analyzing distributed interaction , 2010, Int. J. Comput. Support. Collab. Learn..

[25]  Anouschka van Leeuwen,et al.  Teacher interventions in a synchronous, co-located CSCL setting: Analyzing focus, means, and temporality , 2013, Comput. Hum. Behav..

[26]  G. Stahl.  Rediscovering CSCL , 2001 .

[27]  Lars Bollen,et al.  Creating cognitive tutors for collaborative learning: steps toward realization , 2006, User Modeling and User-Adapted Interaction.

[28]  P. Dillenbourg.  What do you mean by collaborative learning , 1999 .

[29]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[30]  Pierre Dillenbourg,et al.  Over-scripting CSCL: The risks of blending collaborative learning with instructional design , 2002 .

[31]  Jan-Willem Strijbos,et al.  A new direction for log file analysis in CSCL: experiences with a spatio-temporal metric , 2005, CSCL.

[32]  Issa M. Saleh,et al.  NEW SCIENCE OF LEARNING: COGNITION, COMPUTERS AND COLLABORATION IN EDUCATION , 2010 .

[33]  Pierre Dillenbourg,et al.  Technology for Classroom Orchestration , 2010 .

[34]  Gerry Stahl,et al.  Meaning making in CSCL: conditions and preconditions for cognitive processes by groups , 2007, CSCL.

[35]  Heinz Ulrich Hoppe,et al.  Computer supported moderation of e-discussions: the ARGUNAUT approach , 2007, CSCL.

[36]  E. Duval.  Attention please!: learning analytics for visualization and recommendation , 2011, LAK.

[37]  Dennis Zielke,et al.  Design and Implementation of a Learning Analytics Toolkit for Teachers , 2012, J. Educ. Technol. Soc..

[38]  Pierre Dillenbourg,et al.  The Evolution of Research on Computer-Supported Collaborative Learning , 2009 .

[39]  Thomas C. Reeves,et al.  Meaningful interaction in web-based learning: A social constructivist interpretation , 2007, Internet High. Educ..

[40]  N. Avouris,et al.  Rating dimensions of collaboration quality in synchronous collaborating dyads: findings and interpretations , 2011, CSCL.

[41]  Analía Amandi,et al.  Intelligent assistance for teachers in collaborative e-learning environments , 2009, Comput. Educ..

[42]  Carol K. K. Chan.  Bridging research and practice: Implementing and sustaining knowledge building in Hong Kong classrooms , 2011, Int. J. Comput. Support. Collab. Learn..