Quantitative approach to collaborative learning: performance prediction, individual assessment, and group composition

The benefits of collaborative learning, although widely reported, lack the quantitative rigor and detailed insight into the dynamics of interactions within the group, while individual contributions and their impacts on group members and their collaborative work remain hidden behind joint group assessment. To bridge this gap we intend to address three important aspects of collaborative learning focused on quantitative evaluation and prediction of group performance. First, we use machine learning techniques to predict group performance based on the data of member interactions and thereby identify whether, and to what extent, the group’s performance is driven by specific patterns of learning and interaction. Specifically, we explore the application of Extreme Learning Machine and Classification and Regression Trees to assess the predictability of group academic performance from live interaction data. Second, we propose a comparative model to unscramble individual student performances within the group. These performances are then used further in a generative mixture model of group grading as an explicit combination of isolated individual student grade expectations and compared against the actual group performances to define what we coined as collaboration synergy - directly measuring the improvements of collaborative learning. Finally the impact of group composition of gender and skills on learning performance and collaboration synergy is evaluated. The analysis indicates a high level of predictability of group performance based solely on the style and mechanics of collaboration and quantitatively supports the claim that heterogeneous groups with the diversity of skills and genders benefit more from collaborative learning than homogeneous groups.

[1]  Surjeet Kumar Yadav,et al.  Data Mining: A Prediction for Performance Improvement of Engineering Students using Classification , 2012, ArXiv.

[2]  David W. Johnson,et al.  Cooperative Learning and Social Interdependence Theory , 2002 .

[3]  Rui Guo,et al.  Participation-based student final performance prediction model through interpretable Genetic Programming: Integrating learning analytics, educational data mining and theory , 2015, Comput. Hum. Behav..

[4]  Ulrike Cress,et al.  The need for considering multilevel analysis in CSCL research—An appeal for the use of more advanced statistical methods , 2008, Int. J. Comput. Support. Collab. Learn..

[5]  Daniel D. Suthers,et al.  Technology affordances for intersubjective meaning making: A research agenda for CSCL , 2006, Int. J. Comput. Support. Collab. Learn..

[6]  J. Hackman,et al.  Group tasks, group interaction process, and group performance effectiveness: A review and proposed integration , 1975 .

[7]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[8]  Wolfgang Lehmann In a Class of Their Own , 2013 .

[9]  David W. Johnson,et al.  Cooperative learning methods: A meta-analysis. , 2012 .

[10]  Alexander W. Chizhik,et al.  Equity Issues in Collaborative Group Assessment: Group Composition and Performance , 1998 .

[11]  Imad H. Elhajj,et al.  Turning Student Groups into Effective Teams , 2004 .

[12]  Stephanie D. Teasley,et al.  The Construction of Shared Knowledge in Collaborative Problem Solving , 1995 .

[13]  S. Puntambekar,et al.  The Brink of Change: Gender in Technology-Rich Collaborative Learning Environments , 2004 .

[14]  Nguyen Thai-Nghe,et al.  Multi-Relational Factorization Models for Predicting Student Performance , 2011 .

[15]  Erika Johnson,et al.  Learning analytics for collaborative writing: a prototype and case study , 2012, LAK '12.

[16]  Timothy Koschmann,et al.  Dewey's contribution to the foundations of CSCL research , 2002, CSCL.

[17]  Revathi Chennabathni,et al.  Gender Issues in Collaborative Learning , 1998 .

[18]  Amy Bruckman,et al.  Gender and programming achievement in a CSCL environment , 2002, CSCL.

[19]  Jonathan C. Hilpert,et al.  Collaborative Learning in Engineering Students: Gender and Achievement , 2011 .

[20]  César Hervás-Martínez,et al.  Data Mining Algorithms to Classify Students , 2008, EDM.

[21]  Daniel D. Suthers,et al.  Computer-supported collaborative learning: An historical perspective , 2006 .

[22]  Rebecca Ferguson,et al.  Learning analytics to identify exploratory dialogue within synchronous text chat , 2011, LAK.

[23]  Sotiris B. Kotsiantis,et al.  A combinational incremental ensemble of classifiers as a technique for predicting students' performance in distance education , 2010, Knowl. Based Syst..

[24]  Alberto Salguero,et al.  Factors influencing university drop out rates , 2009, Comput. Educ..

[25]  Jan-Willem Strijbos,et al.  Assessment of (Computer-Supported) Collaborative Learning , 2011, IEEE Transactions on Learning Technologies.

[26]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[27]  C. Boxtel,et al.  Collaborative learning tasks and the elaboration of conceptual knowledge. , 2000 .

[28]  Lars Schmidt-Thieme,et al.  Recommender system for predicting student performance , 2010, RecSysTEL@RecSys.

[29]  Leanne Crosswell,et al.  Navigating the muddy waters of the research into single sex class-rooms in co-educational middle years settings. , 2012 .

[30]  N. Webb Collaborative Group Versus Individual Assessment in Mathematics: Processes and Outcomes , 1993 .

[31]  Janet E. Davidson,et al.  The psychology of problem solving , 2003 .

[32]  Manu Kapur,et al.  Productive failure in CSCL groups , 2009, Int. J. Comput. Support. Collab. Learn..

[33]  P. Dillenbourg,et al.  The evolution of research on collaborative learning , 1996 .

[34]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[35]  Teknik Informatika,et al.  PREDICTION OF STUDENT ACADEMIC PERFORMANCE BY AN APPLICATION OF DATA MINING TECHNIQUES , 2011 .

[36]  Lars Schmidt-Thieme,et al.  Personalized Forecasting Student Performance , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[37]  K. Bruffee Collaborative Learning: Higher Education, Interdependence, and the Authority of Knowledge , 1995 .

[38]  Dymitr Ruta,et al.  Does gender matter for collaborative learning? , 2014, 2014 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE).

[39]  R. Bell,et al.  The Effects of Working in Pairs in Science Performance Assessments , 1994 .

[40]  Sean P. Goggins,et al.  Learning analytics at “small” scale , 2015 .

[41]  N. Webb,et al.  Group Discussion and Large-Scale Language Arts Assessment: Effects on Students' Comprehension , 2000 .

[42]  Kinda Khalaf,et al.  Collaborative learning in action , 2013, Proceedings of 2013 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE).

[43]  Regina O. Smith,et al.  Thinking Out of a Bowl of Spaghetti: Learning to Learn in Online Collaborative Groups , 2004 .

[44]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[45]  Lars Schmidt-Thieme,et al.  Improving Academic Performance Prediction by Dealing with Class Imbalance , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[46]  Nguyen Thai Nghe,et al.  A comparative analysis of techniques for predicting academic performance , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[47]  Richard Alterman,et al.  Predicting failure: a case study in co-blogging , 2012, LAK.

[48]  Miguel Nussbaum,et al.  Collaborative robotic instruction: A graph teaching experience , 2009, Comput. Educ..

[49]  Susan Morse,et al.  Separated by sex : a critical look at single-sex education for girls , 1998 .

[50]  H. Phan,et al.  Collaboration: a Literature Review Research Report , 2011 .

[51]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[52]  Chang Zhu,et al.  Student Satisfaction, Performance, and Knowledge Construction in Online Collaborative Learning , 2012, J. Educ. Technol. Soc..

[53]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[54]  Françoise Détienne,et al.  Quality of collaboration in a distant collaborative architectural educational setting , 2010 .

[55]  M. Chiu Flowing Toward Correct Contributions During Group Problem Solving: A Statistical Discourse Analysis , 2008 .

[56]  N. Webb Group Collaboration in Assessment: Multiple Objectives, Processes, and Outcomes , 1995 .

[57]  Saurabh Pal,et al.  Data Mining: A prediction for performance improvement using classification , 2012, ArXiv.

[58]  K. Patricia Cross,et al.  Collaborative Learning Techniques: A Handbook for College Faculty , 2004 .

[59]  Manu Kapur,et al.  A Complexity-Grounded Model for the Emergence of Convergence in CSCL Groups , 2011 .

[60]  Wilhelmiina Hämäläinen,et al.  Classifiers for educational data mining , 2010 .

[61]  P. Dillenbourg What do you mean by collaborative learning , 1999 .

[62]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[63]  N. Webb Task-Related Verbal Interaction and Mathematics Learning in Small Groups. , 1991 .

[64]  Dymitr Ruta,et al.  Learning alone or in a group — An empirical case study of the collaborative learning patterns and their impact on student grades , 2014, 2014 International Conference on Interactive Collaborative Learning (ICL).

[65]  David W. Johnson,et al.  Cooperative learning : increasing college faculty instructional productivity , 1991 .

[66]  R. Slavin Cooperative Learning: Theory, Research and Practice , 1990 .

[67]  Robert E. Slavin,et al.  Improving Intergroup Relations: Lessons Learned From Cooperative Learning Programs , 1999 .

[68]  Lars Schmidt-Thieme,et al.  Factorization Models for Forecasting Student Performance , 2011, EDM.

[69]  Mike Sharkey,et al.  Course correction: using analytics to predict course success , 2012, LAK '12.

[70]  V. Savicki,et al.  Gender and group composition in small task groups using computer-mediated communication , 1996 .

[71]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[72]  Susan Brown Feichtner,et al.  Why Some Groups Fail: a Survey of Students' Experiences with Learning Groups , 1984 .

[73]  Linda Corrin,et al.  Visualizing patterns of student engagement and performance in MOOCs , 2014, LAK.

[74]  Glauco De Vita Fostering intercultural learning through multicultural group work , 2007 .

[75]  Sebastián Ventura,et al.  Predicting students' final performance from participation in on-line discussion forums , 2013, Comput. Educ..

[76]  Fred A. Mael,et al.  Single-Sex Versus Coeducational Schooling: A Systematic Review. Doc # 2005-01. , 2005 .

[77]  Amir Zeid,et al.  Impact of introducing single-gender classrooms in higher education on student achievement levels: A case study in software engineering courses in the GCC region , 2011, 2011 Frontiers in Education Conference (FIE).

[78]  Ronghuai Huang,et al.  The effects of sentiments and co-regulation on group performance in computer supported collaborative learning , 2016, Internet High. Educ..

[79]  Guido Caicedo,et al.  Online Collaboration: Individual Involvement Used to Predict Team Performance , 2014, HCI.

[80]  Rachel A. Lotan,et al.  Can Groups Learn , 2002 .

[81]  樊希强,et al.  Cooperative Learning , 2020, Teaching High School Physics.

[82]  Allyson Hadwin,et al.  Measurement and assessment in computer-supported collaborative learning , 2010, Comput. Hum. Behav..

[83]  Sean P. Goggins,et al.  Learning analytics in CSCL with a focus on assessment: an exploratory study of activity theory-informed cluster analysis , 2014, LAK.

[84]  Ming Ming Chiu,et al.  Group Problem-Solving Processes: Social Interactions and Individual Actions , 2000 .

[85]  Jeff Golub,et al.  Focus on collaborative learning , 1989 .