Setting the pace: examining cognitive processing in MOOC discussion forums with automatic text analysis

ABSTRACT Learning analytics focuses on extracting meaning from large amounts of data. One of the largest datasets in education comes from Massive Open Online Courses (MOOCs) that typically feature enrollments in the tens of thousands. Analyzing MOOC discussion forums presents logistical issues, resulting chiefly from the size of the dataset, which can create challenges for understanding and adequately describing student behaviors. Utilizing automatic text analysis, this study built a hierarchical linear model that examines the influence of the pacing condition of a massive open online course (MOOC), whether it is self-paced or instructor-paced, on the demonstration of cognitive processing in a HarvardX MOOC. The analysis of 2,423 discussion posts generated by 671 students revealed the number of dictionary words used were positively associated with cognitive processing while analytical thinking and clout was negatively associated. We found that none of the student background information (gender, education), status of the course engagement (explored or completed), or the course pace (self-paced versus instructor paced) significantly influenced the cognitive processing of the postings.

[1]  Thomas K. F. Chiu,et al.  Factors influencing peer learning and performance in MOOC asynchronous online discussion forum , 2017, Australasian Journal of Educational Technology.

[2]  Betsy Duke,et al.  Connectivism as a Learning Theory for the Digital Age , 2012 .

[3]  Robert E. Mercer,et al.  Writing to persuade: Analysis and detection of persuasive discourse , 2017 .

[4]  Claire Howell Major,et al.  Massive Open Online Courses: Variations on a New Instructional Form. , 2016 .

[5]  Curtis J. Bonk,et al.  Content analysis of online discussion in an applied educational psychology course , 2000 .

[6]  K. Hew,et al.  Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges , 2014 .

[7]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[8]  Helen Christensen,et al.  A Linguistic Analysis of Suicide-Related Twitter Posts , 2017, Crisis.

[9]  Charles R. Graham,et al.  Measuring student engagement in technology-mediated learning: A review , 2015, Comput. Educ..

[10]  Erica B. Slotter,et al.  Finding the silver lining , 2015 .

[11]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[12]  Rebecca Jen-Hui Wang,et al.  Automated Text Analysis for Consumer Research , 2018 .

[13]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[14]  Aimée A. Kane,et al.  Language and Group Processes: An Integrative, Interdisciplinary Review , 2018, Small Group Research.

[15]  Cassandra L. Carlson,et al.  The Language of Extremity , 2016 .

[16]  D. Mcloughlin,et al.  An analysis of higher order thinking in online discussions , 2009 .

[17]  Yurgos Politis,et al.  Involving People with Autism in Development of Virtual World for Provision of Skills Training , 2017 .

[18]  Afsaneh Sharif,et al.  Discussion Forums in MOOCs , 2015 .

[19]  Jane Sinclair,et al.  Exploring the use of MOOC discussion forums , 2014 .

[20]  Hedieh Najafi,et al.  A Comparison of Learner Intent and Behaviour in Live and Archived MOOCs. , 2014 .

[21]  Zehra Akyol,et al.  Understanding cognitive presence in an online and blended community of inquiry: Assessing outcomes and processes for deep approaches to learning , 2011, Br. J. Educ. Technol..

[22]  Zahide Yildirim,et al.  Investigation of community of inquiry framework in regard to self-regulation, metacognition and motivation , 2018, Comput. Educ..

[23]  J. Pennebaker,et al.  Lying Words: Predicting Deception from Linguistic Styles , 2003, Personality & social psychology bulletin.

[24]  Michael S. Bernstein,et al.  Lexicons on Demand: Neural Word Embeddings for Large-Scale Text Analysis , 2017, IJCAI.

[25]  Badrul H. Khan,et al.  Responsible Analytics and Data Mining in Education : Global Perspectives on Quality, Support, and Decision Making , 2018 .

[26]  Joan L. Whipp,et al.  Scaffolding Critical Reflection in Online Discussions , 2003 .

[27]  Ryan L. Boyd,et al.  The Development and Psychometric Properties of LIWC2015 , 2015 .

[28]  Anat Cohen,et al.  Analysing successful massive open online courses using the community of inquiry model as perceived by students , 2018, J. Comput. Assist. Learn..

[29]  Katrina A. Meyer,et al.  FACE-TO-FACE VERSUS THREADED DISCUSSIONS: THE ROLE OF TIME AND HIGHER-ORDER THINKING , 2019, Online Learning.

[30]  J. Pennebaker,et al.  Linguistic Markers of Psychological Change Surrounding September 11, 2001 , 2004, Psychological science.

[31]  Arthur C. Graesser,et al.  Modeling Learners' Cognitive, Affective, and Social Processes through Language and Discourse , 2014, J. Learn. Anal..

[32]  Tony R. Martinez,et al.  Detecting Cognitive Distortions Through Machine Learning Text Analytics , 2017, 2017 IEEE International Conference on Healthcare Informatics (ICHI).

[33]  Chareen Snelson,et al.  Teaching Massive, Open, Online, Courses (MOOCs): Tales from the Front Line , 2018, The International Review of Research in Open and Distributed Learning.

[34]  R. M. Tobin,et al.  Measuring emotional expression with the Linguistic Inquiry and Word Count. , 2007, The American journal of psychology.

[35]  Jennifer Beckmann,et al.  Cognitive presence in virtual collaborative learning: Assessing and improving critical thinking in online discussion forums , 2015, Interact. Technol. Smart Educ..

[36]  Bernard J. Jansen,et al.  An Analysis of MOOC Discussion Forum Interactions from the Most Active Users , 2015, SBP.

[37]  Jim Hewitt,et al.  Exploring asynchronous and synchronous tool use in online courses , 2013, Comput. Educ..

[38]  Robert L. Moore,et al.  The Role of Data Analytics in Education , 2018, Responsible Analytics and Data Mining in Education.

[39]  H. Pham,et al.  Weigh your words: An exploration of natural word use in a fat studies course , 2018 .

[40]  Jens Breivik,et al.  Critical Thinking in Online Educational Discussions Measured as Progress through Inquiry Phases: A Discussion of the Cognitive Presence Construct in the Community of Inquiry Framework , 2016 .

[41]  D. Garrison,et al.  Critical thinking, cognitive presence, and computer conferencing in distance education , 2001 .

[42]  Achraf Touati.  Self-Directed Learning in MOOCs: A Disconnect Between Theory and Practice , 2016 .

[43]  Sarah Hayes,et al.  MOOCs and quality: a review of the recent literature , 2015 .

[44]  George Veletsianos,et al.  A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015 , 2016 .

[45]  Timothy Savage,et al.  Comparing learner community behavior in multiple presentations of a Massive Open Online Course , 2016, J. Comput. High. Educ..

[46]  Karen Swan,et al.  A Response to the Review of the Community of Inquiry Framework , 2009 .

[47]  A. Graesser,et al.  Pronoun Use Reflects Standings in Social Hierarchies , 2014 .

[48]  Kristy Elizabeth Boyer,et al.  Unsupervised modeling for understanding MOOC discussion forums: a learning analytics approach , 2015, LAK.

[49]  D. Garrison.  ONLINE COMMUNITY OF INQUIRY REVIEW: SOCIAL, COGNITIVE, AND TEACHING PRESENCE ISSUES , 2019, Online Learning.

[50]  John Nerbonne,et al.  The Secret Life of Pronouns. What Our Words Say About Us , 2014, Lit. Linguistic Comput..

[51]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[52]  Robert F. Boruch,et al.  Moving Through MOOCs , 2014 .

[53]  Peter Shea,et al.  Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learning environments , 2010, Comput. Educ..

[54]  J. Dewey.  How we think : a restatement of the relation of reflective thinking to the educative process , 1934 .

[55]  Yin Yin,et al.  A phenomenology of learning large: the tutorial sphere of xMOOC video lectures , 2014 .

[56]  George Siemens,et al.  Towards automated content analysis of discussion transcripts: a cognitive presence case , 2016, LAK.

[57]  Sheizaf Rafaeli,et al.  Interactivity in online discussions and learning outcomes , 2016, Comput. Educ..

[58]  Tobias Hecking,et al.  Uncovering the structure of knowledge exchange in a MOOC discussion forum , 2015, 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[59]  E. Xing,et al.  Towards an Integration of Text and Graph Clustering Methods as a Lens for Studying Social Interaction in MOOCs , 2014 .

[60]  Robert Moore,et al.  The Role of Data Analytics in Education: Possibilities & Limitations , 2018 .

[61]  Qiyun Wang,et al.  Investigating critical thinking and knowledge construction in an interactive learning environment , 2009, Interact. Learn. Environ..

[62]  William Ickes,et al.  Predicting Final Course Performance From Students’ Written Self-Introductions , 2013 .

[63]  D. Randy Garrison,et al.  Cognitive presence in online learning , 2004, J. Comput. High. Educ..

[64]  R. Shields,et al.  MOOCs, disruptive innovation and the future of higher education: A conceptual analysis , 2019 .

[65]  Katy Jordan,et al.  Massive Open Online Course Completion Rates Revisited: Assessment, Length and Attrition , 2015 .

[66]  M. Graves,et al.  Using the LIWC Program to Study Dreams , 2018 .

[67]  Qiang Yang,et al.  Students Motivation in Asynchronous Online Discussions with MOOC Mode , 2014 .

[68]  William A. Donohue,et al.  Validating LIWC Dictionaries , 2014 .