ReaderBench: Automated evaluation of collaboration based on cohesion and dialogism

As Computer-Supported Collaborative Learning (CSCL) gains a broader usage, the need for automated tools capable of supporting tutors in the time-consuming process of analyzing conversations becomes more pressing. Moreover, collaboration, which presumes the intertwining of ideas or points of view among participants, is a central element of dialogue performed in CSCL environments. Therefore, starting from dialogism and a cohesion-based model of discourse, we propose and validate two computational models for assessing collaboration. The first model is based on a cohesion graph and can be perceived as a longitudinal analysis of the ongoing conversation, thus accounting for collaboration from a social knowledge-building perspective. In the second approach, collaboration is regarded from a dialogical perspective as the intertwining or synergy of voices pertaining to different speakers, therefore enabling a transversal analysis of subsequent discussion slices.

[1]  Andrei Popescu-Belis,et al.  What are discourse markers ? , 2003 .

[2]  Rudolf Stephan,et al.  The New Harvard Dictionary of Music , 1988 .

[3]  Kathleen McKeown,et al.  Improving Word Sense Disambiguation in Lexical Chaining , 2003, IJCAI.

[4]  Ulrike Cress,et al.  Mass Collaboration and Learning , 2013 .

[5]  Amanda Stent,et al.  Annotating Argumentation Acts in Spoken Dialog , 2000 .

[6]  Carl Bereiter,et al.  Education and Mind in the Knowledge Age , 2002 .

[7]  F. Fischer,et al.  A framework to analyze argumentative knowledge construction in computer-supported collaborative learning , 2006, Comput. Educ..

[8]  Traian Rebedea,et al.  Visualisation of Learners' Contributions in Chat Conversations , 2007 .

[9]  Per Linell,et al.  Dialogue in Focus Groups: Exploring Socially Shared Knowledge , 2007 .

[10]  Stefan Trausan-Matu,et al.  Cohesion-based Analysis of CSCL Conversations: Holistic and Individual Perspectives , 2013, CSCL.

[11]  G. Savova,et al.  An Introduction to Natural Language Processing: How You Can Get More From Those Electronic Notes You Are Generating. , 2015, Pediatric emergency care.

[12]  Danielle S. McNamara,et al.  Predicting Comprehension from Students' Summaries , 2015, AIED.

[13]  Csr Young,et al.  How to Do Things With Words , 2009 .

[14]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[15]  Traian Rebedea,et al.  PolyCAFe : Collaboration and Utterance Assessment for Online CSCL Conversations The Core Concepts of Bakhtin ’ s Dialogism , 2011 .

[16]  Stefan Trausan-Matu,et al.  ReaderBench, an Environment for Analyzing Text Complexity and Reading Strategies , 2013, AIED.

[17]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[18]  Arthur C. Graesser,et al.  Coh-Metrix: Capturing Linguistic Features of Cohesion , 2010 .

[19]  M. Bakhtin,et al.  Speech genres and other late essays , 1986 .

[20]  Stefan Trausan-Matu,et al.  Mining Texts, Learner Productions and Strategies with ReaderBench , 2014 .

[21]  Rupert Wegerif,et al.  A dialogic understanding of the relationship between CSCL and teaching thinking skills , 2005, CSCL.

[22]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[23]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[24]  Stefan Trausan-Matu,et al.  Automatic Support for the Analysis of Online Collaborative Learning Chat Conversations , 2010, ICHL.

[25]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[26]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[27]  Graeme Hirst,et al.  Evaluating WordNet-based Measures of Lexical Semantic Relatedness , 2006, CL.

[28]  Haim Levkowitz,et al.  Introduction to information retrieval (IR) , 2008 .

[29]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[30]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[31]  Daniel D. Suthers,et al.  Using a Contingency Graph to Discover Representational Practices in an Online Collaborative Environment , 2009, Res. Pract. Technol. Enhanc. Learn..

[32]  M. Bakhtin,et al.  Problems of Dostoevsky's Poetics , 1985 .

[33]  Danielle S. McNamara,et al.  Discourse cohesion, a signature of collaboration , 2019 .

[34]  Stefan Trausan-Matu,et al.  Polyphonic Support for Collaborative Learning , 2006, CRIWG.

[35]  Craig H. Martell,et al.  Topic Detection and Extraction in Chat , 2008, 2008 IEEE International Conference on Semantic Computing.

[36]  Victor Kaptelinin,et al.  Group Cognition Computer Support for Building Collaborative Knowledge , 2007 .

[37]  Carolyn Penstein Rosé,et al.  Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning , 2008, Int. J. Comput. Support. Collab. Learn..

[38]  David Eichelberger,et al.  Speech Genres And Other Late Essays , 2016 .

[39]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[40]  M. Scardamalia,et al.  Knowledge Building: Theory, Pedagogy, and Technology , 2006 .

[41]  Stefan Gerry Johann Trausan-Matu,et al.  Supporting Polyphonic Collaborative Learning , 2007 .

[42]  Traian Rebedea,et al.  Polyphonic Inter-Animation of Voices in VMT , 2009 .

[43]  April Kontostathis,et al.  Comparison of Rule-based to Human Analysis of Chat Logs , 2009 .

[44]  Michael Halliday,et al.  Cohesion in English , 1976 .

[45]  Virginia Teller Review of Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition by Daniel Jurafsky and James H. Martin. Prentice Hall 2000. , 2000 .

[46]  Rob Nadolski,et al.  Scripted collaboration in serious gaming for complex learning: Effects of multiple perspectives when acquiring water management skills , 2011, Br. J. Educ. Technol..

[47]  Andy Dong,et al.  Concept formation as knowledge accumulation: A computational linguistics study , 2006, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[48]  M. Scardamalia Collective cognitive responsibility for the advancement of knowledge , 2002 .

[49]  Lev Vygotsky Mind in society , 1978 .

[50]  Arthur C. Graesser,et al.  Automated Evaluation of Text and Discourse with Coh-Metrix: Introduction , 2014 .

[51]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994 .

[52]  Stefan Trausan-Matu,et al.  Validating the Automated Assessment of Participation and of Collaboration in Chat Conversations , 2014, Intelligent Tutoring Systems.

[53]  Stefan Trausan-Matu,et al.  Polyphonic Inter-Animation of Voices in Chats , 2007 .

[54]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[55]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[56]  Krishna P. Gummadi,et al.  Measurement and analysis of online social networks , 2007, IMC '07.

[57]  Arthur C. Graesser,et al.  Automated Evaluation of Text and Discourse with Coh-Metrix: Coh-Metrix Measures , 2014 .

[58]  M. Bakhtin,et al.  The Dialogic Imagination: Four Essays , 1981 .

[59]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[60]  Daniel D. Suthers,et al.  Exposing Chat Features through Analysis of Uptake between Contributions , 2012, 2012 45th Hawaii International Conference on System Sciences.

[61]  Traian Rebedea,et al.  A Deep Insight in Chat Analysis: Collaboration, Evolution and Evaluation, Summarization and Search , 2010, AIMSA.

[62]  Andy Dong,et al.  The latent semantic approach to studying design team communication , 2005 .

[63]  A. Koller,et al.  Speech Acts: An Essay in the Philosophy of Language , 1969 .

[64]  Per Linell Rethinking Language, Mind and World Dialogically : Interactional and contextual theories of human sense-making , 2009 .

[65]  Timothy Koschmann,et al.  Toward a dialogic theory of learning: Bakhtin's contribution to understanding learning in settings of collaboration , 1999, CSCL.

[66]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[67]  Torsten Holmer,et al.  Explicit Referencing in Learning Chats: Needs and Acceptance , 2006, EC-TEL.

[68]  Heeyoung Lee,et al.  A Multi-Pass Sieve for Coreference Resolution , 2010, EMNLP.

[69]  Daniel D. Suthers,et al.  Computer-supported collaborative learning: An historical perspective , 2006 .

[70]  M. Cole,et al.  Mind in society: The development of higher psychological processes. L. S. Vygotsky. , 1978 .

[71]  Isabelle Tapiero,et al.  Situation Models and Levels of Coherence: Toward a Definition of Comprehension , 2007 .

[72]  Stefan Trausan-Matu,et al.  Voices' inter-animation detection with readerbench modelling and assessing polyphony in CSCL chats as voice synergy , 2013, 2nd International Conference on Systems and Computer Science.

[73]  Traian Rebedea,et al.  PolyCAFe—automatic support for the polyphonic analysis of CSCL chats , 2014, International Journal of Computer-Supported Collaborative Learning.

[75]  John Goodier,et al.  A Dictionary of Statistics , 2003 .

[76]  Stefan Trausan-Matu,et al.  Textual Complexity and Discourse Structure in Computer-Supported Collaborative Learning , 2012, ITS.

[77]  Danielle S. McNamara,et al.  Dialogism: A Framework for CSCL and a Signature of Collaboration , 2015, CSCL.

[78]  Mihai Dascalu,et al.  Analyzing Discourse and Text Complexity for Learning and Collaborating - A Cognitive Approach Based on Natural Language Processing , 2013, Studies in Computational Intelligence.

[79]  Janet McDonnell,et al.  The Language of Design: Theory and Computation , 2010 .

[80]  D. Randel The New Harvard Dictionary of Music , 1988 .

[81]  Traian Rebedea,et al.  A System for the Automatic Analysis of Computer-Supported Collaborative Learning Chats , 2012, 2012 IEEE 12th International Conference on Advanced Learning Technologies.

[82]  Tracey Potts,et al.  Rhythmanalysis: Space, Time and Everyday Life , 2015 .

[83]  Daniel Jurafsky,et al.  An introduction to natural language processing , 2000 .

[84]  Traian Rebedea,et al.  A Polyphonic Model and System for Inter-animation Analysis in Chat Conversations with Multiple Participants , 2010, CICLing.