Multiphase method for analysing online discussions

Several studies have analysed and assessed online performance and discourse using quantitative and qualitative methods. Quantitative measures have typically included the analysis of participation rates and learning outcomes in terms of grades. Qualitative measures of postings, discussions and context features aim to give insights into the nature of learning activities taking place in the online learning environment. No matter what the particular method, one of the critical aspects in these analyses is the coherence between theory and methodology as well as the explicit focus on the unit(s) of analysis. Technology-enhanced learning (TEL) settings typically involve processes (e.g. cognitive or social) at different levels, and these processes are often intertwined with each other in a way that is not reducible to any one level only. In this article, a short review on the challenges of research-based evaluation of TEL, and computer-supported collaborative learning in particular, is made. With these challenges in the background, the integration of different levels of analysis is demonstrated with the aid of the method developed for analysing the level and reciprocity of discussions in a Web-based learning setting.

[1]  Daniel D. Suthers,et al.  A framework for conceptualizing, representing, and analyzing distributed interaction , 2010, Int. J. Comput. Support. Collab. Learn..

[2]  Michael J. Baker,et al.  Rainbow: a framework for analysing computer-mediated , 2007 .

[3]  M. Chi.  Quantifying Qualitative Analyses of Verbal Data: A Practical Guide , 1997 .

[4]  K. Kumpulainen,et al.  The situated dynamics of peer group interaction: an introduction to an analytic framework , 1999 .

[5]  Raija Hämäläinen.  Methodological reflections: designing and understanding computer-supported collaborative learning , 2012 .

[6]  Sadhana Puntambekar,et al.  Analyzing collaborative interactions: divergence, shared understanding and construction of knowledge , 2006, Comput. Educ..

[7]  Hans Christian Arnseth,et al.  Approaching institutional contexts: systemic versus dialogic research in CSCL , 2006, Int. J. Comput. Support. Collab. Learn..

[8]  G. Stahl,et al.  Methodological issues in developing a multi-dimensional coding procedure for small-group chat communication , 2007 .

[9]  Tammy Schellens,et al.  Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review , 2006, Comput. Educ..

[10]  S. Järvelä,et al.  Combining Individual and Group-Level Perspectives for Studying Collaborative Knowledge Construction in Context. , 2007 .

[11]  M. Scardamalia,et al.  Designs for Collective Cognitive Responsibility in Knowledge-Building Communities , 2009 .

[12]  L. Morris,et al.  Tracking student behavior, persistence, and achievement in online courses , 2005, Internet High. Educ..

[13]  Päivi Häkkinen,et al.  Sharing and constructing perspectives in web-based conferencing , 2006, Comput. Educ..

[14]  Jan-Willem Strijbos,et al.  Methodological challenges for collaborative learning research , 2007 .

[15]  Nikol Rummel,et al.  A rating scheme for assessing the quality of computer-supported collaboration processes , 2007, Int. J. Comput. Support. Collab. Learn..

[16]  Päivi Häkkinen,et al.  Specifying computer-supported collaboration scripts , 2007, Int. J. Comput. Support. Collab. Learn..

[17]  Carol K. K. Chan,et al.  Student-Directed Assessment of Knowledge Building Using Electronic Portfolios , 2007 .

[18]  Carlos Caldeira,et al.  Group Cognition: Computer Support for Building Collaborative Knowledge (Acting with Technology) , 2006 .

[19]  Liam Rourke,et al.  Validity in quantitative content analysis , 2004 .

[20]  Brigid Barron.  When Smart Groups Fail , 2003 .

[21]  Cindy E. Hmelo-Silver,et al.  Analyzing collaborative knowledge construction: multiple methods for integrated understanding , 2003, Comput. Educ..

[22]  Peter Reimann,et al.  Time is precious: Variable- and event-centred approaches to process analysis in CSCL research , 2009, Int. J. Comput. Support. Collab. Learn..

[23]  Tammy Schellens,et al.  Collaborative learning in asynchronous discussion groups: What about the impact on cognitive processing? , 2005, Comput. Hum. Behav..

[24]  Teresa Mauri,et al.  Toward a dialectic relation between the results in CSCL: Three critical methodological aspects of content analysis schemes , 2010, Int. J. Comput. Support. Collab. Learn..

[25]  Cindy E. Hmelo-Silver,et al.  Understanding collaborative learning processes in new learning environments , 2008 .

[26]  K. Hakkarainen.  Emergence of Progressive-Inquiry Culture in Computer-Supported Collaborative Learning , 2003 .

[27]  F. Fischer,et al.  Knowledge convergence in collaborative learning: Concepts and assessment , 2007 .

[28]  Päivi Häkkinen,et al.  Shared and personal learning spaces: Challenges for pedagogical design , 2012, Internet High. Educ..

[29]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[30]  Maarit Arvaja,et al.  Personal and shared experiences as resources for meaning making in a philosophy of science course , 2012, Int. J. Comput. Support. Collab. Learn..

[31]  Fatos Xhafa,et al.  What's in the mix? Combining coding and conversation analysis to investigate chat-based problem solving , 2007 .

[32]  F. Fischer,et al.  A framework to analyze argumentative knowledge construction in computer-supported collaborative learning , 2006, Comput. Educ..

[33]  Kai Hakkarainen,et al.  Effective Participation and Discourse through a Computer Network: Investigating Elementary Students' Computer Supported Interaction , 2002 .

[34]  Gerry Stahl,et al.  Paradigms of shared knowledge , 2009, Int. J. Comput. Support. Collab. Learn..

[35]  Scripting Computer-Supported Collaborative Learning: A Review of SCORE Studies , 2010 .

[36]  Sanna Järvelä,et al.  Analyzing CMC content for what? , 2006, Comput. Educ..