Group Learning Assessment: Developing a Theory-Informed Analytics

* Corresponding author ABSTRACT Assessment in Computer Supported Collaborative Learning (CSCL) is an implicit issue, and most assessments are summative in nature. Process-oriented methods of assessment can vary significantly in their indicators and typically only partially address the complexity of group learning. Moreover, the majority of these assessment methods require time-intensive coding of qualitative data. Our study explores the operationalization of activity theory to frame group activity in a CSCL context by breaking group work into six dimensions. We map log data generated by a collaborative software, Virtual Math Teams with Geogebra, with these dimensions and construct six measures to lay the groundwork for automating CSCL assessment. Next, we move beyond identification and analysis of those measures to infer group learning using human judgment and employ a clustering algorithm to categorize groups with similar performance, allowing us to consider the six indicators simultaneously and to step further toward assessment automation. Last, in terms of the complexity of group learning in a socio- technical context, we discuss a web-based tool that not only shows group-level assessment but also integrates with our previous work on individual assessment, thus providing teachers with a diverse of group learning.

[1]  Antonio Fini,et al.  Visualizing and monitoring effective interactions in online collaborative groups , 2010, Br. J. Educ. Technol..

[2]  Tariq M. Khan,et al.  The relationship between educational performance and online access routines: analysis of students' access to an online discussion forum , 2012, LAK '12.

[3]  H. Garfinkel.  Studies in Ethnomethodology , 1968 .

[4]  Stefan Hrastinski,et al.  A theory of online learning as online participation , 2009, Comput. Educ..

[5]  Françoise Détienne,et al.  Quality of collaboration in a distant collaborative architectural educational setting , 2010 .

[6]  Xin Chen,et al.  "Twitter Archeology" of learning analytics and knowledge conferences , 2015, LAK.

[7]  A. Sfard.  On Two Metaphors for Learning and the Dangers of Choosing Just One , 1998 .

[8]  Amy Soller,et al.  Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning , 2004, User Modeling and User-Adapted Interaction.

[9]  L. S. Vygotksy.  Mind in society: the development of higher psychological processes , 1978 .

[10]  Peter Reimann,et al.  Time is precious: Variable- and event-centred approaches to process analysis in CSCL research , 2009, Int. J. Comput. Support. Collab. Learn..

[11]  Austin Henderson,et al.  Interaction Analysis: Foundations and Practice , 1995 .

[12]  Jan-Willem Strijbos,et al.  Assessment of (Computer-Supported) Collaborative Learning , 2011, IEEE Transactions on Learning Technologies.

[13]  Chang Zhu,et al.  Student Satisfaction, Performance, and Knowledge Construction in Online Collaborative Learning , 2012, J. Educ. Technol. Soc..

[14]  Allyson Hadwin,et al.  Measurement and assessment in computer-supported collaborative learning , 2010, Comput. Hum. Behav..

[15]  Sean P. Goggins,et al.  Learning analytics in CSCL with a focus on assessment: an exploratory study of activity theory-informed cluster analysis , 2014, LAK.

[16]  Rui Guo,et al.  Identifying Time-of-Day Breakpoints Based on Nonintrusive Data Collection Platforms , 2014, J. Intell. Transp. Syst..

[17]  Rui Guo,et al.  Decision Support Based on Time-Series Analytics: A Cluster Methodology , 2014, HCI.

[18]  Gaea Leinhardt,et al.  Functions, Graphs, and Graphing: Tasks, Learning, and Teaching , 1990 .

[19]  M. Chaib,et al.  ICT in Teacher Education : Challenging prospects , 2005 .

[20]  Vive S. Kumar,et al.  Assessing process in CSCL: An ontological approach , 2010, Comput. Hum. Behav..

[21]  Wu Yong-he,et al.  Ontology construction and mathematical modeling for the LAS engines , 2014 .

[22]  Carlos Caldeira,et al.  Group Cognition: Computer Support for Building Collaborative Knowledge (Acting with Technology) , 2006 .

[23]  G. Stahl.  Theories of Cognition in Collaborative Learning , 2013 .

[24]  Manu Kapur,et al.  Productive failure in CSCL groups , 2009, Int. J. Comput. Support. Collab. Learn..

[25]  Rui Guo,et al.  Google Analytics based Temporal-Geospatial Analysis for Web Management: A Case Study of a K-12 Online Resource Website , 2014 .

[26]  Y. Engeström,et al.  Learning by expanding: An activity-theoretical approach to developmental research , 2014 .

[27]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[28]  Rui Guo,et al.  Participation-based student final performance prediction model through interpretable Genetic Programming: Integrating learning analytics, educational data mining and theory , 2015, Comput. Hum. Behav..

[29]  Vic Lally,et al.  Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for Social Network Analysis , 2007, Int. J. Comput. Support. Collab. Learn..

[30]  Wei Wang,et al.  Evaluation of the impacts of traffic states on crash risks on freeways. , 2012, Accident; analysis and prevention.

[31]  Gerry Stahl,et al.  Translating Euclid: Designing a Human-Centered Mathematics , 2013, Translating Euclid: Designing a Human-Centered Mathematics.

[32]  Lara Lomicka,et al.  Assessing online collaboration among language teachers: A cross-institutional case study , 2009 .

[33]  N. Frederiksen.  The real test bias: Influences of testing on teaching and learning. , 1984 .

[34]  L. Lipponen,et al.  Patterns of participation and discourse in elementary students’ computer-supported collaborative learning , 2003 .

[35]  J. Ola Lindberg,et al.  Sharing the distance or a distance shared : Social and individual aspects of participation in ICT-supported distance-based teacher education , 2005 .

[36]  Fatos Xhafa,et al.  A layered framework for evaluating on-line collaborative learning interactions , 2006, Int. J. Hum. Comput. Stud..

[37]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[38]  Angelique Dimitracopoulou,et al.  Graphical Interaction Analysis Impact on Groups Collaborating through Blogs , 2013, J. Educ. Technol. Soc..

[39]  Marcelo Milrad,et al.  Using Construction Kits, Modeling Tools, System Dynamics Simulations to Support Collaborative Discovery Learning , 2002, J. Educ. Technol. Soc..

[40]  Sean P. Goggins,et al.  Learning analytics in outer space: a Hidden Naïve Bayes model for automatic student off-task behavior detection , 2015, LAK.

[41]  Sean P. Goggins,et al.  Learning analytics at “small” scale , 2015 .