Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification

Short text classification is a fundamental task in natural language processing. It is hard due to the lack of context information and labeled data in practice. In this paper, we propose a new method called SHINE, which is based on graph neural network (GNN), for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs which introduce more semantic and syntactic information. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts. Thus, comparing with existing GNN-based methods, SHINE can better exploit interactions between nodes of the same types and capture similarities between short texts. Extensive experiments on various benchmark short text datasets show that SHINE consistently outperforms state-of-the-art methods, especially with fewer labels.1

[1]  Sundararajan Sellamanickam,et al.  HeteGCN: Heterogeneous Graph Convolutional Networks for Text Classification , 2020, WSDM.

[2]  Houfeng Wang,et al.  Text Level Graph Neural Network for Text Classification , 2019, EMNLP.

[3]  Linmei Hu,et al.  Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification , 2019, EMNLP.

[4]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[5]  Xuanjing Huang,et al.  Recurrent Neural Network for Text Classification with Multi-Task Learning , 2016, IJCAI.

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  Charu C. Aggarwal,et al.  A Survey of Text Classification Algorithms , 2012, Mining Text Data.

[8]  Jin Wang,et al.  Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification , 2017, IJCAI.

[9]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[10]  Yu Chen,et al.  Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings , 2019, NeurIPS.

[11]  Huan Liu,et al.  Be More with Less: Hypergraph Attention Networks for Inductive Text Classification , 2020, EMNLP.

[12]  Haopeng Zhang,et al.  Text Graph Transformer for Document Classification , 2020, EMNLP.

[13]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[14]  Susumu Horiguchi,et al.  Learning to classify short and sparse text & web with hidden topics from large-scale data collections , 2008, WWW.

[15]  Quanming Yao,et al.  Search to aggregate neighborhood for graph neural network , 2021, 2021 IEEE 37th International Conference on Data Engineering (ICDE).

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[18]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[19]  Xuanjing Huang,et al.  Contextualized Non-local Neural Networks for Sequence Learning , 2018, AAAI.

[20]  Liqiang Nie,et al.  HGAT: Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification , 2021, ACM Trans. Inf. Syst..

[21]  Jing Li,et al.  Topic Memory Networks for Short Text Classification , 2018, EMNLP.

[22]  Markus Krötzsch,et al.  Wikidata , 2014, Commun. ACM.

[23]  Ji Wu,et al.  Tensor Graph Convolutional Networks for Text Classification , 2020, AAAI.

[24]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[25]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[26]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  M. de Rijke,et al.  Short Text Similarity with Word Embeddings , 2015, CIKM.

[29]  Philip S. Yu,et al.  A Survey on Text Classification: From Traditional to Deep Learning , 2020, ACM Trans. Intell. Syst. Technol..

[30]  Yufeng Zhang,et al.  Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks , 2020, ACL.

[31]  Isabelle Guyon,et al.  Taking Human out of Learning Applications: A Survey on Automated Machine Learning , 2018, 1810.13306.

[32]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[33]  Zhiyong Li,et al.  Document and Word Representations Generated by Graph Convolutional Network and BERT for Short Text Classification , 2020, ECAI.

[34]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[35]  Jindong Chen,et al.  Deep Short Text Classification with Knowledge Powered Attention , 2019, AAAI.