Hybrid Attention Networks for Chinese Short Text Classification

To improve the classification performance for Chinese short text with automatic semantic feature selection, in this paper we propose the Hybrid Attention Networks (HANs) which combines the word- and character-level selective attentions. The model firstly applies RNN and CNN to extract the semantic features of texts. Then it captures class-related attentive representation from word- and character-level features. Finally, all of the features are concatenated and fed into the output layer for classification. Experimental results on 32-class and 5-class datasets show that, our model  outperforms multiple baselines by combining not only the word- and character-level features of the texts, but also class-related semantic features by attentive mechanism.

[1]  Xuanjing Huang,et al.  Multi-Timescale Long Short-Term Memory Neural Network for Modelling Sentences and Documents , 2015, EMNLP.

[2]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[3]  Zhiyuan Liu,et al.  Statistical and semantic analysis of rumors in Chinese social media , 2015 .

[4]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[5]  Changliang Li,et al.  Compositional Recurrent Neural Networks for Chinese Short Text Classification , 2016, 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Mengen Chen,et al.  Short Text Classification Improved by Learning Multi-Granularity Topics , 2011, IJCAI.

[9]  Sarah Zelikovitz,et al.  Transductive Learning For Short-Text Classification Problems Using Latent Semantic Indexing , 2005, Int. J. Pattern Recognit. Artif. Intell..

[10]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[11]  Wenjie Li,et al.  Component-Enhanced Chinese Character Embeddings , 2015, EMNLP.

[12]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[13]  Wei Liang,et al.  Chinese Short Text Classification Based on Domain Knowledge , 2013, IJCNLP.

[14]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[15]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[16]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[17]  Hugo Jair Escalante,et al.  Distributional Term Representations for Short-Text Categorization , 2013, CICLing.

[18]  Yong Zhang,et al.  Attention pooling-based convolutional neural network for sentence modelling , 2016, Inf. Sci..

[19]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[20]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[21]  Xiaolin Du,et al.  Short Text Classification: A Survey , 2014, J. Multim..