A C-LSTM with Attention Mechanism for Question Categorization

Text categorization plays a vital role in text mining applications such as customer care service, business intelligence, and records management. Specifically, text categorization assigns content into a set of predefined aspects or categories. In this paper, we propose a convolutional long short-term memory (C-LSTM) with an attention mechanism for question categorization. The convolutional neural network (CNN) layer extracts higher-level feature representation on input data. These features fed into an LSTM network for creating a document or sentence context representation sequentially. Attention mechanism employed to pay selective attention to the output of C-LSTM. Also, a fully connected layer and output layer added on the top of the attention layer. We evaluate the proposed model on the question classification dataset. The results show that the proposed C-LSTM with attention mechanism outperforms.

[1]  Lei Huang,et al.  Novel architecture for long short-term memory used in question classification , 2018, Neurocomputing.

[2]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[3]  Peng Zhou,et al.  Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling , 2016, COLING.

[4]  Gaurav Aggarwal,et al.  Enhancing Web Search Through Question Classifier , 2018 .

[5]  Han Zhao,et al.  Self-Adaptive Hierarchical Sentence Model , 2015, IJCAI.

[6]  Youngjoong Ko,et al.  Efficient question classification and retrieval using category information and word embedding on cQA services , 2019, Journal of Intelligent Information Systems.

[7]  Tevfik Aytekin,et al.  Real-time recommendation with locality sensitive hashing , 2019, Journal of Intelligent Information Systems.

[8]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[9]  Xiang Li,et al.  Densely Connected Bidirectional LSTM with Applications to Sentence Classification , 2018, NLPCC.

[10]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[11]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[12]  Xuanjing Huang,et al.  Question Classification using Multiple Classifiers , 2005, ALR/ALRN@IJCNLP.

[13]  Luísa Coheur,et al.  From symbolic to sub-symbolic information in question classification , 2011, Artificial Intelligence Review.

[14]  Zengchang Qin,et al.  Question Classification using Head Words and their Hypernyms , 2008, EMNLP.

[15]  Mark Lee,et al.  High Accuracy Rule-based Question Classification using Question Syntax and Semantics , 2016, COLING.

[16]  Yang Liu,et al.  Deep Learning in Machine Translation , 2018 .

[17]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[18]  Yang Liu,et al.  An Attention-Gated Convolutional Neural Network for Sentence Classification , 2018, Intell. Data Anal..

[19]  Xiaoyong Du,et al.  Initializing Convolutional Filters with Semantic Features for Text Classification , 2017, EMNLP.

[20]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[21]  Zhonghe He,et al.  Question Classification Based on Hybrid Neural Networks , 2016 .

[22]  Abeer Alsadoon,et al.  Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review , 2019, Expert Syst. Appl..

[23]  Suresh Manandhar,et al.  Dependency Based Embeddings for Sentence Classification Tasks , 2016, NAACL.

[24]  Ruoyu Chen,et al.  A Text Sentiment Classification Modeling Method Based on Coordinated CNN‐LSTM‐Attention Model , 2019, Chinese Journal of Electronics.

[25]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[26]  Dan Roth,et al.  Learning Question Classifiers , 2002, COLING.

[27]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[28]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[29]  Hossein Nezamabadi-pour,et al.  Multilabel feature selection: A comprehensive review and guiding experiments , 2018, Wiley Interdiscip. Rev. Data Min. Knowl. Discov..