Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks

Text classification is fundamental in natural language processing (NLP), and Graph Neural Networks (GNN) are recently applied in this task. However, the existing graph-based works can neither capture the contextual word relationships within each document nor fulfil the inductive learning of new words. In this work, to overcome such problems, we propose TextING for inductive text classification via GNN. We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures, which can also effectively produce embeddings for unseen words in the new document. Finally, the word nodes are aggregated as the document embedding. Extensive experiments on four benchmark datasets show that our method outperforms state-of-the-art text classification methods.

[1]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[2]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[3]  Christina Lioma,et al.  Graph-based term weighting for information retrieval , 2011, Information Retrieval.

[4]  Tieniu Tan,et al.  GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction , 2019, Pattern Recognit..

[5]  Xuanjing Huang,et al.  Recurrent Neural Network for Text Classification with Multi-Task Learning , 2016, IJCAI.

[6]  Houfeng Wang,et al.  Text Level Graph Neural Network for Text Classification , 2019, EMNLP.

[7]  Georgios Paliouras,et al.  An evaluation of Naive Bayesian anti-spam filtering , 2000, ArXiv.

[8]  Xing Xie,et al.  Session-based Recommendation with Graph Neural Networks , 2018, AAAI.

[9]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[10]  George Forman,et al.  BNS feature scaling: an improved representation over tf-idf for svm text classification , 2008, CIKM '08.

[11]  Songbo Tan,et al.  An effective refinement strategy for KNN text classifier , 2006, Expert Syst. Appl..

[12]  LiomaChristina,et al.  Graph-based term weighting for information retrieval , 2012 .

[13]  Tieniu Tan,et al.  Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification , 2019, IJCAI.

[14]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[15]  Michalis Vazirgiannis,et al.  Text Categorization as a Graph Classification Problem , 2015, ACL.

[16]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[17]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[18]  Liang Wang,et al.  Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction , 2019, CIKM.

[19]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[20]  Michalis Vazirgiannis,et al.  Message Passing Attention Networks for Document Understanding , 2019, AAAI.

[21]  Guoyin Wang,et al.  Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms , 2018, ACL.

[22]  TanSongbo An effective refinement strategy for KNN text classifier , 2006 .

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.