R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning

The task of concept prerequisite chain learning is to automatically determine the existence of prerequisite relationships among concept pairs. In this paper, we frame learning prerequisite relationships among concepts as an unsupervised task with no access to labeled concept pairs during training. We propose a model called the Relational-Variational Graph AutoEncoder (R-VGAE) to predict concept relations within a graph consisting of concept and resource nodes. Results show that our unsupervised approach outperforms graph-based semi-supervised methods and other baseline methods by up to 9.77% and 10.47% in terms of prerequisite relation prediction accuracy and F1 score. Our method is notably the first graph-based model that attempts to make use of deep learning representations for the task of unsupervised prerequisite learning. We also expand an existing corpus which totals 1,717 English Natural Language Processing (NLP)-related lecture slide files and manual concept pair annotations over 322 topics.

[1]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[2]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[3]  Timothy Baldwin,et al.  Automatic Labelling of Topics with Neural Embeddings , 2016, COLING.

[4]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[5]  C. Lee Giles,et al.  Investigating Active Learning for Concept Prerequisite Learning , 2018, AAAI.

[6]  Chengjiang Li,et al.  Prerequisite Relation Learning for Concepts in MOOCs , 2017, ACL.

[7]  Eneko Agirre,et al.  Unsupervised Statistical Machine Translation , 2018, EMNLP.

[8]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[9]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[10]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[11]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[12]  Dragomir R. Radev,et al.  What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learning , 2018, AAAI.

[13]  Premkumar Natarajan,et al.  Modeling Concept Dependencies in a Scientific Corpus , 2016, ACL.

[14]  Zhaohui Wu,et al.  Recovering Concept Prerequisite Relations from University Course Dependencies , 2017, AAAI.

[15]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[16]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[17]  Yiming Yang,et al.  Data-driven Automated Induction of Prerequisite Structure Graphs , 2016, EDM.

[18]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[19]  Aitor García Pablos,et al.  W2VLDA: Almost unsupervised system for Aspect Based Sentiment Analysis , 2017, Expert Syst. Appl..

[20]  Dragomir R. Radev,et al.  TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation , 2018, ACL.

[21]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[22]  Luiz Chaimowicz,et al.  Learning Transferable Features For Open-Domain Question Answering , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[23]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[24]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[25]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[26]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[27]  ChengXiang Zhai,et al.  Mining MOOC Lecture Transcripts to Construct Concept Dependency Graphs , 2018, EDM.

[28]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[29]  Yiming Yang,et al.  Learning Concept Graphs from Online Educational Data , 2016, J. Artif. Intell. Res..

[30]  Peter Brusilovsky,et al.  Semi-Supervised Techniques for Mining Learning Outcomes and Prerequisites , 2017, KDD.

[31]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.