Content analysis: What are they talking about?

Quantitative content analysis is increasingly used to surpass surface level analyses in computer-supported collaborative learning (e.g., counting messages), but critical reflection on accepted practice has generally not been reported. A review of CSCL conference proceedings revealed a general vagueness in definitions of units of analysis. In general, arguments for choosing a unit were lacking and decisions made while developing the content analysis procedures were not made explicit. In this article, it will be illustrated that the currently accepted practices concerning the 'unit of meaning' are not generally applicable to quantitative content analysis of electronic communication. Such analysis is affected by 'unit boundary overlap' and contextual constraints having to do with the technology used. The analysis of e-mail communication required a different unit of analysis and segmentation procedure. This procedure proved to be reliable, and the subsequent coding of these units for quantitative analysis yielded satisfactory reliabilities. These findings have implications and recommendations for current content analysis practice in CSCL research.

[1]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[2]  A. Madill,et al.  Objectivity and reliability in qualitative analysis: realist, contextualist and radical constructionist epistemologies. , 2000, British journal of psychology.

[3]  Jan-Willem Strijbos,et al.  Designing for interaction: Six steps to designing computer-supported group-based learning , 2004, Comput. Educ..

[4]  Vic Lally,et al.  Complexity, theory and praxis: Researching collaborative learning and tutoring processes in a networked learning community , 2003 .

[5]  Gerry Stahl,et al.  Computer Support for Collaborative Learning : foundations for a CSCL community : proceedings of CSCL 2002, Boulder, Colorado, USA, January 7-11, 2002 , 2002 .

[6]  Kai Hakkarainen,et al.  Computer support for participatory designing: a pilot study , 2002, CSCL.

[7]  Minjuan Wang,et al.  The Construction of Shared Knowledge in an Internet-based Shared Environment for Expeditions (iExpeditions) , 2001 .

[8]  W. Jochems,et al.  The Effect of Functional Roles on Group Efficiency , 2004 .

[9]  Kai Pata,et al.  Framework for Scaffolding the Development of Problem Representations by Collaborative Design , 2003, CSCL.

[10]  Vic Lally,et al.  A Quartet in E , 2003, CSCL.

[11]  Lucia Rapanotti,et al.  Mapping interactions in a computer conferencing environment , 2001 .

[12]  Jim Hewitt.  How Habitual Online Practices Affect the Development of Asynchronous Discussion Threads , 2003 .

[13]  David Traum,et al.  Interlocution Scenarios for Problem Solving in an Educational MUD Environment , 2001 .

[14]  E. Wong,et al.  Developmental Trajectory in Knowledge Building: An Investigation , 2003, CSCL.

[15]  L. Lipponen,et al.  Analyzing patterns of participation and discourse in elementary students ' online science discussion , 2004 .

[16]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[17]  Paul A. Kirschner,et al.  Validating a Representational Notation for Collaborative Problem Solving , 2003, CSCL.

[18]  Michael J. Baker,et al.  Computer-Supported Collaborative Learning in the Space of Debate , 2003, CSCL.

[19]  Alessandra Talamo,et al.  Examining synchronous tutoring in a virtual world , 2002, CSCL.

[20]  Liam Rourke,et al.  Validity in quantitative content analysis , 2004 .

[21]  Ingrid Mulder,et al.  Stimulating Questioning Behaviour , 2003, CSCL.

[22]  Starr Roxanne Hiltz,et al.  Learning Networks: A Field Guide to Teaching and Learning Online , 1995 .

[23]  Linda Harasim,et al.  Collaborating in Cyberspace: Using Computer Conferences as a Group Learning Environment , 1993, Interact. Learn. Environ..

[24]  M. Chi.  Quantifying Qualitative Analyses of Verbal Data: A Practical Guide , 1997 .

[25]  Timothy Koschmann,et al.  Cscl : Theory and Practice of An Emerging Paradigm , 1996 .

[26]  P. E. Mudrack,et al.  An Examination of Functional Role Behavior and Its Consequences for Individuals in Group Settings , 1995 .

[27]  Barbara Wasson,et al.  Collaboration and Problem Solving in Distributed Collaborative Learning , 2001 .

[28]  F. Fischer,et al.  Fostering collaborative knowledge construction with visualization tools , 2002 .

[29]  Ckk Chan,et al.  Beyond “sitting next to each other”: A design experiment on knowledge building in teacher education , 2001 .

[30]  Annemarie Hauf,et al.  Computers in education , 1983 .

[31]  Jan-Willem Strijbos,et al.  What we know about CSCL: ...and what we do not (but need to) know about CSCL , 2004 .

[32]  Christiaan Hamaker,et al.  Een nieuwe bijdrage tot validatie van het (meta)cognitieve deel van de Inventaris Leerstijlen , 1997 .

[33]  Ion Juvina,et al.  Role of Icons and Chat Boxes in Computer Supported Collaborative Learning , 2003, CSCL.

[34]  A. Hare,et al.  Types of Roles in Small Groups , 1994 .

[35]  Jan-Willem Strijbos,et al.  Computer-Supported Collaboration in Small Groups The Effect of Functional Roles on Group Efficiency : Using Multilevel Modeling and Content Analysis to , 2004 .

[36]  Lars Svensson,et al.  Interaction repertoire in a distance education community , 2002, CSCL.

[37]  L. Lipponen,et al.  Patterns of participation and discourse in elementary students’ computer-supported collaborative learning , 2003 .

[38]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[39]  H. Muukkonen,et al.  Characteristics of university students ’ inquiry in individual and computer-supported collaborative study process , 2004 .

[40]  Timothy Koschmann,et al.  Paradigm shifts and instructional technology : An introduction , 1996 .

[41]  Nikolaos M. Avouris,et al.  A Study on Heterogeneity During Realtime Collaborative Problem Solving , 2003, CSCL.

[42]  F. Fischer,et al.  A framework to analyze argumentative knowledge construction in computer-supported collaborative learning , 2006, Comput. Educ..

[43]  Pierre Dillenbourg,et al.  What we know about CSCL and implementing it in higher education , 2004 .

[44]  R. Elliott,et al.  Evolving guidelines for publication of qualitative research studies in psychology and related fields. , 1999, The British journal of clinical psychology.

[45]  Martin Valcke,et al.  Asynchrone discussiegroepen: een onderzoek naar de invloed op cognitieve kennisverwerking. , 2002 .

[46]  Tammy Schellens,et al.  Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review , 2006, Comput. Educ..

[47]  Päivi Häkkinen,et al.  Is successful scaffolding an illusion? – Shifting patterns of responsibility and control in teacher-student interaction during a long-term learning project , 2003 .

[48]  Curtis J. Bonk,et al.  Content analysis of online discussion in an applied educational psychology course , 2000 .

[49]  Charlotte N. Gunawardena,et al.  Analysis of a Global Online Debate and the Development of an Interaction Analysis Model for Examining Social Construction of Knowledge in Computer Conferencing , 1997 .

[50]  Starr Roxanne Hiltz,et al.  Impacts of Asynchronous Learning Networks on Individual and Group Problem Solving: A Field Experiment , 1999 .

[51]  Daniel D. Suthers,et al.  Deictic Roles of External Representations in Face-to-Face and Online Collaboration , 2003, CSCL.

[52]  Sten R. Ludvigsen,et al.  Designing for Change in Networked Learning Environments (Computer-Supported Collaborative Learning Series) , 2004 .

[53]  D. Newman,et al.  A Content Analysis Method to Measure Critical Thinking in Face-to-Face and Computer Supported Group Learning , 1995 .

[54]  A. E. Veldhuis-Diermanse.  CSCLearning? Participation, learning activities and knowledge construction in computer-supported collaborative learning in higher education (Summary PhD dissertation) , 2002 .

[55]  Jan-Willem Strijbos,et al.  The effect of roles on computer-supported collaborative learning. , 2004 .

[56]  Elizabeth A. Lenell,et al.  Evaluating affordance short-circuits by reviewers and authors participating in on-line journal reviews , 2001 .

[57]  David Hammer,et al.  Students' Collaborative use of Computer-Based Programming Tools in Science , 2003, CSCL.

[58]  Frances Slack,et al.  The development of deep learning during a synchronous collaborative on-line course , 2002, CSCL.

[59]  Patrick J. Fahy,et al.  Patterns of Interaction in a Computer Conference Transcript , 2001 .

[60]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[61]  Jerry Andriessen,et al.  Comparing Graphical and Textual Preparation Tools for Collaborative Argumentation-Based Learning , 2003, CSCL.

[62]  D. Garrison,et al.  Methodological Issues in the Content Analysis of Computer Conference Transcripts , 2007 .

[63]  Gijsbert Erkens,et al.  Supporting Historical Reasoning in CSCL , 2003, CSCL.

[64]  Daniel C. A. Hillman,et al.  A New Method for Analyzing Patterns of Interaction , 1999 .

[65]  Gerry Stahl,et al.  Contributions to a theoretical framework for CSCL , 2002, CSCL.

[66]  Vic Lally,et al.  Cracking the code: learning to collaborate and collaborating to learn in a networked environment , 2002, CSCL.

[67]  Maarten De Laat,et al.  Network and content analysis in an online community discourse , 2002, CSCL.