Language and Discourse Analysis with Coh-Metrix: Applications from Educational Material to Learning Environments at Scale

The goal of this article is to preserve and distribute the information presented at the LASI (2014) workshop on Coh-Metrix, a theoretically grounded, computational linguistics facility that analyzes texts on multiple levels of language and discourse. The workshop focused on the utility of Coh-Metrix in discourse theory and educational practice. We discuss some of the motivating factors that led to the development of Coh-Metrix, situated within the context of multilevel theoretical frameworks of discourse comprehension and learning. A review of published studies will highlight the applications of Coh-Metrix, ranging from the scaling and selection of educational material to learning environments at scale. The examples illustrate the relationship between discourse and cognitive, affective, and social processes. We walk through the pedagogical guidelines that should be followed when analyzing texts using Coh-Metrix. Finally, we conclude the paper with a general discussion of the future directions for Coh-Metrix including methodological and practical implications for the learning analytics (LA) and educational data mining (EDM) community.

[1]  Michael Halliday,et al.  Cohesion in English , 1976 .

[2]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[3]  D. Tannen.  Spoken and written language : exploring orality and literacy , 1984 .

[4]  Morton Ann Gernsbacher,et al.  Language Comprehension As Structure Building , 1990 .

[5]  T. Trabasso,et al.  Constructing inferences during narrative text comprehension. , 1994, Psychological review.

[6]  T. Trabasso,et al.  Minimal or Global Inference during Reading , 1994 .

[7]  Rolf A. Zwaan,et al.  The Construction of Situation Models in Narrative Comprehension: An Event-Indexing Model , 1995 .

[8]  W. Kintsch,et al.  Are Good Texts Always Better? Interactions of Text Coherence, Background Knowledge, and Levels of Understanding in Learning From Text , 1996 .

[9]  A. Jackson Stenner,et al.  Measuring Reading Comprehension with the Lexile Framework. , 1996 .

[10]  Danielle S. McNamara,et al.  Learning from texts: Effects of prior knowledge and text coherence , 1996 .

[11]  Rolf A. Zwaan,et al.  Discourse comprehension. , 1997, Annual review of psychology.

[12]  Morton Ann Gernsbacher,et al.  Two Decades of Structure Building. , 1997, Discourse processes.

[13]  Walter Kintsch,et al.  Comprehension: A Paradigm for Cognition , 1998 .

[14]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[15]  Robert F. Lorch,et al.  Memory-Based Text Processing: Assumptions and Issues. , 1998 .

[16]  T. Dalgleish,et al.  Handbook of cognition and emotion , 1999 .

[17]  Hyun-Jeong Joyce Kim,et al.  Updating Situation Models From Descriptive Texts: A Test of the Situational Operator Model , 2000 .

[18]  Ted Sanders,et al.  The Role of Coherence Relations and Their Linguistic Markers in Text Processing , 2000 .

[19]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[20]  Walter Kintsch,et al.  Text Retrieval: A Theoretical Exploration , 2001 .

[21]  M. Louwerse.  An analytic and cognitve parameterization of coherence relations , 2002 .

[22]  A. Graesser,et al.  Handbook of discourse processes , 2003 .

[23]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[24]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[25]  M. Cole,et al.  Mind in society: The development of higher psychological processes. L. S. Vygotsky. , 1978 .

[26]  Arthur C. Graesser,et al.  Question asking and eye tracking during cognitive disequilibrium: Comprehending illustrated texts on devices when the devices break down , 2005, Memory & cognition.

[27]  Arthur C. Graesser,et al.  Discourse cohesion in text and tutorial dialogue , 2007 .

[28]  David N. Rapp,et al.  Higher-Order Comprehension Processes in Struggling Readers: A Perspective for Research and Intervention , 2007 .

[29]  Danielle S. McNamara,et al.  Reversing the Reverse Cohesion Effect: Good Texts Can Be Better for Strategic, High-Knowledge Readers , 2007 .

[30]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[31]  Scott Jarvis,et al.  vocd: A theoretical and empirical evaluation , 2007 .

[32]  Joseph P. Magliano,et al.  Chapter 9 Toward a Comprehensive Model of Comprehension , 2009 .

[33]  Arthur C. Graesser,et al.  Cohesion Relationships in Tutorial Dialogue as Predictors of Affective States , 2009, AIED.

[34]  Ivan K. Ash,et al.  Source Evaluation, Comprehension, and Learning in Internet Science Inquiry Tasks , 2009 .

[35]  Gerry Stahl,et al.  From Individual Representations to Group Cognition , 2009 .

[36]  Sidney K. D'Mello,et al.  The Intricate Dance between Cognition and Emotion during Expert Tutoring , 2010, Intelligent Tutoring Systems.

[37]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[38]  Arthur C. Graesser,et al.  Coh-Metrix: Capturing Linguistic Features of Cohesion , 2010 .

[39]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[40]  Arthur C. Graesser,et al.  Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features , 2010, User Modeling and User-Adapted Interaction.

[41]  Rebecca Saxe,et al.  fMRI item analysis in a theory of mind task , 2011, NeuroImage.

[42]  Arthur C. Graesser,et al.  Computational Analyses of Multilevel Discourse Comprehension , 2011, Top. Cogn. Sci..

[43]  Andrew Elfenbein.  Research in Text and the Uses of Coh-Metrix , 2011 .

[44]  Arthur C. Graesser,et al.  Coh-Metrix , 2011 .

[45]  Arthur C. Graesser,et al.  Language and Discourse Are Powerful Signals of Student Emotions during Tutoring , 2012, IEEE Transactions on Learning Technologies.

[46]  Sidney K. D'Mello,et al.  Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies , 2012, ICMI '12.

[47]  Jason L. G. Braasch,et al.  Comprehending and Learning From Internet Sources: Processing Patterns of Better and Poorer Learners , 2012, Reading Research Quarterly.

[48]  Arthur C. Graesser,et al.  Unimodal and Multimodal Human Perceptionof Naturalistic Non-Basic Affective Statesduring Human-Computer Interactions , 2013, IEEE Transactions on Affective Computing.

[49]  Arthur C. Graesser,et al.  Coh-Metrix Measures Text Characteristics at Multiple Levels of Language and Discourse , 2014, The Elementary School Journal.

[50]  A. Graesser,et al.  Confusion can be beneficial for learning. , 2014 .

[51]  Arthur C. Graesser,et al.  The psychology of science text comprehension , 2014 .

[52]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[53]  Arthur C. Graesser,et al.  What Works: Creating Adaptive and Intelligent Systems for Collaborative Learning Support , 2014, Intelligent Tutoring Systems.

[54]  George Siemens,et al.  Where is research on massive open online courses headed? A data analysis of the MOOC research initiative , 2014 .

[55]  Arthur C. Graesser,et al.  Automated Evaluation of Text and Discourse with Coh-Metrix: Introduction , 2014 .

[56]  Arthur C. Graesser,et al.  Modeling Student Socioaffective Responses to Group Interactions in a Collaborative Online Chat Environment , 2014, EDM.

[57]  Arthur C. Graesser,et al.  How do you connect?: analysis of social capital accumulation in connectivist MOOCs , 2015, LAK.

[58]  Arthur C. Graesser,et al.  Modeling Learners' Social Centrality and Performance through Language and Discourse , 2015, EDM.

[59]  Mike Rinck,et al.  Narrative Comprehension , Causality , and Coherence ~ ~ Essays in Honor of Tom Trabasso , 2022 .