Detecting cognitive engagement using word embeddings within an online teacher professional development community

Abstract Research states that effective teacher professional development (PD) engages teachers as active learners and co-creators of content. However, it is yet to be known whether such pedagogy impacts on cognitive engagement. We adopt the ICAP Framework to measure cognitive engagement in a teacher PD Massively Open Online Course (MOOC). We use word embeddings to automate the identification of teachers' community contributions as representing ‘active’ engagement by manipulating course materials, or ‘constructive’ engagement through the generation of new knowledge. We explored individual variation in engagement across units. Our findings demonstrated that the participants' cognitive engagement is influenced by the nature of MOOC tasks. We adopted a manual content analysis approach to explore constructive contributions. From 67 cases considered, all but one case was identified as containing ‘constructive knowledge’, providing a solid basis for replicating our proposed methodology to analyse cognitive engagement within the community-centric MOOC models.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[3]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[4]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[5]  George Siemens,et al.  Examining communities of inquiry in Massive Open Online Courses: The role of study strategies , 2019, Internet High. Educ..

[6]  Carolyn Penstein Rosé,et al.  Towards triggering higher-order thinking behaviors in MOOCs , 2016, LAK.

[7]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[8]  Brit Toven-Lindsey,et al.  Virtually unlimited classrooms: Pedagogical practices in massive open online courses , 2015, Internet High. Educ..

[9]  Arthur C. Graesser,et al.  Automated Evaluation of Text and Discourse with Coh-Metrix: List of Tables , 2014 .

[10]  Danielle S. McNamara,et al.  Language to Completion: Success in an Educational Data Mining Massive Open Online Class , 2015, EDM.

[11]  George Siemens,et al.  Exploring communities of inquiry in Massive Open Online Courses , 2018, Comput. Educ..

[12]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[13]  Shieh Ruey,et al.  A case study of constructivist instructional strategies for adult online learning , 2010, Br. J. Educ. Technol..

[14]  D. Randy Garrison,et al.  Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education , 1999, Internet High. Educ..

[15]  M. Chi,et al.  The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes , 2014 .

[16]  J. B. Arbaugh,et al.  An Investigation of Epistemological and Social Dimensions of Teaching in Online Learning Environments , 2006 .

[17]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[18]  Arthur C. Graesser,et al.  Newness and givenness of information: Automated identification in written discourse , 2012 .

[19]  Arthur C. Graesser,et al.  Using LSA to Automatically Identify Givenness and Newness of Noun Phrases in Written Discourse , 2005 .

[20]  Nickolas J. G. Falkner,et al.  Reflecting on Three Offerings of a Community-Centric MOOC for K-6 Computer Science Teachers , 2017, SIGCSE.

[21]  Fan-Chuan Tseng,et al.  A study of social participation and knowledge sharing in the teachers' online professional community of practice , 2014, Comput. Educ..

[22]  Thushari Atapattu,et al.  Automatic Detection of Cross-Disciplinary Knowledge Associations , 2018, ACL.

[23]  Etienne Wenger,et al.  Communities of Practice: Learning, Meaning, and Identity , 1998 .

[24]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[25]  Fei Gao,et al.  Exploring the relationship between online discourse and commitment in Twitter professional learning communities , 2018, Comput. Educ..

[26]  Mary Ann Wolf,et al.  The Digital Learning Transition MOOC for Educators: Exploring a Scalable Approach to Professional Development , 2013 .

[27]  Olusola Adesope,et al.  Measuring Engineering Students’ In-class Cognitive Engagement: Survey Development informed by Contemporary Educational Theories , 2018 .

[28]  M. J. Norušis,et al.  SPSS 13.0 Guide to Data Analysis , 2000 .

[29]  Linda Corrin,et al.  Predicting success: how learners' prior knowledge, skills and activities predict MOOC performance , 2015, LAK.

[30]  Dragan Gasevic,et al.  The Changing Patterns of MOOC Discourse , 2017, L@S.

[31]  Tommi S. Jaakkola,et al.  Word, graph and manifold embedding from Markov processes , 2015, ArXiv.

[32]  Diana Laurillard,et al.  The educational problem that MOOCs could solve: professional development for teachers of disadvantaged students , 2016 .

[33]  Alyssa Friend Wise,et al.  Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling , 2017, Internet High. Educ..

[34]  Hsiu-Mei Huang,et al.  Toward constructivism for adult learners in online learning environments , 2002, Br. J. Educ. Technol..

[35]  Kristy Elizabeth Boyer,et al.  Deconstructing the Discussion Forum: Student Questions and Computer Science Learning , 2017, SIGCSE.

[36]  Arthur C. Graesser,et al.  Exploring development of social capital in a CMOOC through language and discourse , 2018, Internet High. Educ..

[37]  Carolyn Penstein Rosé,et al.  Investigating How Student's Cognitive Behavior in MOOC Discussion Forum Affect Learning Gains , 2015, EDM.