Chinese Text Classification Based on Hybrid Model of CNN and LSTM

Text classification is one of the basic tasks of natural language processing. In recent years, deep learning has been widely used in text classification tasks. The representative one is the convolutional neural network. The convolutional neural network(CNN) is limited by the size of the local window and can only extract local features of the text. For long texts like news, CNN cannot learn the longterm dependence of the long text. Another model of deep learning recurrent neural networks based on long short-term memory (LSTM) can learn the long-term dependence of text. Therefore, in the work of this paper, combining the advantages of CNN and LSTM, a LSTM_CNN Hybrid model is constructed for Chinese news text classification tasks. We first use LSTM to learn the longterm dependence of text, then we design a shallow convolution structure to further extract the semantic features of the text, and finally use the max-pooling operation to filter to obtain important features for classification. The model we proposed has achieved good results on the News dataset.

[1]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Regina Barzilay,et al.  Molding CNNs for text: non-linear, non-consecutive convolutions , 2015, EMNLP.

[4]  Yann LeCun,et al.  Very Deep Convolutional Networks for Text Classification , 2016, EACL.

[5]  Jin Wang,et al.  Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification , 2017, IJCAI.

[6]  BengioYoshua,et al.  A neural probabilistic language model , 2003 .

[7]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[8]  Ani Nenkova,et al.  Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2016, NAACL 2016.

[9]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[10]  Tong Zhang,et al.  Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding , 2015, NIPS.

[11]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[12]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[13]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[14]  Tong Zhang,et al.  Deep Pyramid Convolutional Neural Networks for Text Categorization , 2017, ACL.