Term-based pooling in convolutional neural networks for text classification

To achieve good results in convolutional neural networks (CNN) for text classification task, term-based pooling operation in CNNs is proposed. Firstly, the convolution results of several convolution kernels are combined by this method, and then the results after combination are made pooling operation, three sorts of CNN models (we named TB-CNN, MCT-CNN and MMCT-CNN respectively) are constructed and then corresponding algorithmic thought are detailed on this basis. Secondly, relevant experiments and analyses are respectively designed to show the effects of three key parameters (convolution kernel, combination kernel number and word embedding) on three kinds of CNN models and to further demonstrate the effect of the models proposed. The experimental results show that compared with the traditional method of text classification in CNNs, term-based pooling method is addressed that not only the availability of the way is proved, but also the performance shows good superiority.

[1]  Octavian-Eugen Ganea,et al.  Neural Multi-step Reasoning for Question Answering on Semi-structured Tables , 2017, ECIR.

[2]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[3]  Franck Dernoncourt,et al.  Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks , 2016, NAACL.

[4]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[5]  Yong Zhang,et al.  Attention pooling-based convolutional neural network for sentence modelling , 2016, Inf. Sci..

[6]  Alexander M. Rush,et al.  Character-Aware Neural Language Models , 2015, AAAI.

[7]  Ming Zhou,et al.  Question Answering over Freebase with Multi-Column Convolutional Neural Networks , 2015, ACL.

[8]  Jun Chen,et al.  Pedestrian attributes recognition in surveillance scenarios with hierarchical multi-task CNN models , 2018, China Communications.

[9]  Erik Cambria,et al.  Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis , 2015, EMNLP.

[10]  Fernando Enríquez,et al.  An approach to the use of word embeddings in an opinion classification task , 2016, Expert Syst. Appl..

[11]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[12]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Chunxia Zhang,et al.  Answer Extraction Based on Merging Score Strategy of Hot Terms , 2016 .

[15]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[16]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[17]  Rui Dai,et al.  Classifying medical relations in clinical text via convolutional neural networks , 2018, Artif. Intell. Medicine.

[18]  Shaoguo Wen,et al.  Fine-Grained Vehicle Classification With Channel Max Pooling Modified CNNs , 2019, IEEE Transactions on Vehicular Technology.

[19]  Xingshu Chen,et al.  Labeling malicious communication samples based on semi-supervised deep neural network , 2019, China Communications.

[20]  Regina Barzilay,et al.  Molding CNNs for text: non-linear, non-consecutive convolutions , 2015, EMNLP.

[21]  Wenpeng Yin,et al.  Multichannel Variable-Size Convolution for Sentence Classification , 2015, CoNLL.

[22]  Ausif Mahmood,et al.  Convolutional Recurrent Deep Learning Model for Sentence Classification , 2018, IEEE Access.

[23]  Wenpeng Yin,et al.  Attention-Based Convolutional Neural Network for Machine Comprehension , 2016, ArXiv.