Evaluating Collaborative Learning Processes using System-based Measurement

Much of the research on collaborative work focuses on the quality of the group outcome as a measure of success. There is less research on the collaboration process itself, but an understanding of the process should help to improve both the process and the outcomes of collaboration. Understanding and analyzing collaborative learning processes requires a fine-grained analysis of group interaction in the context of learning goals. Taking into account the relationships among tasks, products and collaboration this paper presents a set of measures designed to evaluate the collaborative learning process. We emphasise: direct system-based measures based on data produced by a collaborative learning system during the collaboration process, and suggest that these measures can be enhanced by also considering participants’ perceptions of the process.

[1]  Hans Spada,et al.  Learning in Humans and Machines: Towards an Interdisciplinary Learning Science , 1995 .

[2]  P. Dillenbourg.  What do you mean by collaborative learning , 1999 .

[3]  Spencer Kagan,et al.  The Structural Approach to Cooperative Learning. , 1990 .

[4]  G. Stahl.  Rediscovering CSCL , 2001 .

[5]  José A. Pino,et al.  Towards formal evaluation of collaborative work , 2006, Inf. Res..

[6]  M. Hamm,et al.  Cooperative Learning: Critical Thinking and Collaboration Across the Curriculum. (Second Edition). , 1990 .

[7]  Claire O'Malley,et al.  Computer Supported Collaborative Learning , 1995, NATO ASI Series.

[8]  Alan M. Lesgold,et al.  Knowledge Acquisition for Adaptive Collaborative Learning Environments , 2000 .

[9]  Daniel D. Suthers,et al.  Technology affordances for intersubjective learning: a thematic agenda for CSCL , 2005, CSCL.

[10]  Jane E. Klobas,et al.  Evaluation of the efficacy of collaborative learning in face-to-face and computer-supported university contexts , 2006, Comput. Hum. Behav..

[11]  P. Dillenbourg,et al.  The evolution of research on collaborative learning , 1996 .

[12]  Jonathan J. Cadiz,et al.  Coordination, overload and team performance: effects of team communication strategies , 1998, CSCW '98.

[13]  Barbara J. Millis,et al.  Cooperative Learning for Higher Education Faculty , 1997 .

[14]  Michael Simonson,et al.  Equivalency theory and distance education , 1999 .

[15]  Starr Roxanne Hiltz,et al.  Learning Networks: A Field Guide to Teaching and Learning Online , 1995 .

[16]  José A. Pino,et al.  A first step to formally evaluate collaborative work , 1997, GROUP.

[17]  Beatriz Barros,et al.  A Platform for Collaboration Analysis in CSCL. An ontological approach , 2001 .

[18]  Beatriz Barros,et al.  An Approach to Analyse Collaboration When Shared Structured Workspaces Are Used for Carrying out Group Learning Processes , 1999 .

[19]  N. Kerr.  Motivation losses in small groups: a social dilemma analysis , 1983 .

[20]  César A. Collazos,et al.  Collaborative Scenarios to Promote Positive Interdependence among Group Members , 2003, CRIWG.

[21]  H. Garfinkel.  Studies in Ethnomethodology , 1968 .

[22]  Kai Hakkarainen,et al.  Effective Participation and Discourse through a Computer Network: Investigating Elementary Students' Computer Supported Interaction , 2002 .

[23]  Daniel D. Suthers,et al.  A Coached Collaborative Learning Environment for Entity-Relationship Modeling , 2000, Intelligent Tutoring Systems.

[24]  Daniel D. Suthers,et al.  Coaching Web-based Collaborative Learning based on Problem Solution Differences and Participation , 2003, Int. J. Artif. Intell. Educ..

[25]  Stephanie D. Teasley,et al.  The Construction of Shared Knowledge in Collaborative Problem Solving , 1995 .

[26]  Timothy Koschmann,et al.  Computer Support for Collaboration and Learning , 1999 .

[27]  M. Felisa Verdejo,et al.  Collaborative environments for the learning of design: a model and a case study in Domotics , 2006, Comput. Educ..

[28]  Susan C. Herring,et al.  Gender and Democracy in Computer-Mediated Communication , 1995, Computerization and Controversy, 2nd Ed..

[29]  Gerry Stahl,et al.  Groupware Goes to School , 2002, CRIWG.

[30]  Sandra Katz,et al.  The cognitive skill of coaching collaboration , 1999, CSCL.

[31]  R. Slavin.  Synthesis of Research on Cooperative Learning. , 1981 .

[32]  David W. Johnson,et al.  Learning together and alone: Cooperation, competition, and individualization , 1975 .

[33]  Jane Burdett,et al.  Making Groups Work: University Students' Perceptions , 2003 .

[34]  Peter Levin.  Running group projects: dealing with the free-rider problem , 2003 .

[35]  Linda Harasim,et al.  Collaborating in Cyberspace: Using Computer Conferences as a Group Learning Environment , 1993, Interact. Learn. Environ..

[36]  N. Hara.  STUDENT DISTRESS IN A WEB-BASED DISTANCE EDUCATION COURSE , 2000 .

[37]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[38]  Crescencio Bravo,et al.  DomoSim‐TPC: Collaborative problem solving to support the learning of domotical design , 2006, Comput. Appl. Eng. Educ..

[39]  Heinz Ulrich Hoppe,et al.  Computer supported interaction analysis of group problem solving , 1999, CSCL.

[40]  T. Koschmann,et al.  The Video Analyst's Manifesto (or The Implications of Garfinkel's Policies for the Development of a Program of Video Analytic Research within the Learning Sciences) , 2004, ICLS.

[41]  Gwenny Ruël,et al.  Free-riding and team performance in project education , 2003 .

[42]  César A. Collazos,et al.  Evaluating Collaborative Learning Processes , 2002, CRIWG.

[43]  César A. Collazos,et al.  Evaluating cooperation in group work , 2000, Proceedings Sixth International Workshop on Groupware. CRIWG 2000.

[44]  M. Felisa Verdejo,et al.  Collaborative distributed environments for learning design tasks by means of modelling and simulation , 2006, J. Netw. Comput. Appl..

[45]  Yannis A. Dimitriadis,et al.  Studying social aspects of computer-supported collaboration with a mixed evaluation approach , 2002, CSCL.

[46]  Shlomo Sharan,et al.  Group Investigation Expands Cooperative Learning , 1990 .

[47]  David W. Johnson,et al.  Cooperation and Competition: Theory and Research , 1989 .

[48]  Lasse Lipponen,et al.  Exploring foundations for computer-supported collaborative learning , 2002, CSCL.

[49]  Victor Kaptelinin,et al.  Group Cognition Computer Support for Building Collaborative Knowledge , 2007 .

[50]  David W. Johnson,et al.  Learning Together and Alone , 1999 .

[51]  Harvey Sacks,et al.  Lectures on Conversation , 1995 .

[52]  César A. Collazos,et al.  Improving the Use of Strategies in Computer-Supported Collaborative Processes , 2003, CRIWG.

[53]  César A. Collazos,et al.  A method for evaluating computer-supported collaborative learning processes , 2004, Int. J. Comput. Appl. Technol..

[54]  Charles R. Graham,et al.  Computer-Mediated Learning Groups: Benefits and Challenges to Using Groupwork in Online Learning Environments , 2004 .

[55]  Tim S. Roberts.  Self, Peer and Group Assessment in E-Learning. , 2006 .

[56]  César A. Collazos,et al.  COMPUTATIONAL DESIGN PRINCIPLES TO SUPPORT THE MONITORING OF COLLABORATIVE LEARNING PROCESSES , 2004 .

[57]  David W. Johnson,et al.  Advanced Cooperative Learning , 1998 .

[58]  M. Felisa Verdejo,et al.  Providing adaptation and guidance for design learning by problem solving: The design planning approach in DomoSim-TPC environment , 2007, Comput. Educ..

[59]  Jeff Sokolov.  Methodologies for evaluation of collaborative systems , 1999, SIGG.

[60]  Jean Underwood,et al.  Gender Differences in a Cooperative Computer-Based Language Task. , 1990 .

[61]  Fatos Xhafa,et al.  Problems and Opportunities of Learning Together in a Virtual Learning Environment , 2005 .

[62]  Jane Klobas,et al.  Students' psychological responses to a course supported by collaborative learning technologies: Measurement and preliminary results , 2000 .

[63]  N. Kerr,et al.  Dispensability of member effort and group motivation losses: Free-rider effects , 1983 .

[64]  Alan M. Lesgold,et al.  What Makes Peer Interaction Effective? Modeling Effective Communication in an Intelligent CSCL , 1999 .

[65]  L. Li,et al.  Some Refinements on Peer Assessment of Group Projects , 2001 .

[66]  César A. Collazos,et al.  Collaborative learning and creative writing , 2003, Proceedings of the IEEE/LEOS 3rd International Conference on Numerical Simulation of Semiconductor Optoelectronic Devices (IEEE Cat. No.03EX726).

[67]  Alejandra Martínez-Monés,et al.  From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning , 2005, Int. J. Artif. Intell. Educ..

[68]  Marcia C. Linn,et al.  The case for case studies of programming problems , 1992, CACM.