A novel CNN-based method for Question Classification in Intelligent Question Answering

Sentence classification, which is the foundation of the subsequent text-based processing, plays an important role in the intelligent question answering (IQA). Convolutional neural networks (CNN) as a kind of common architecture of deep learning, has been widely used to the sentence classification and achieved excellent performance in open field. However, the class imbalance problems and fuzzy sentence feature problem are common in IQA. With the aim to get better performance in IQA, this paper proposes a simple and effective method by increasing generalization and the diversity of sentence features based on simple CNN. In proposed method, the professional entities could be replaced by placeholders to improve the performance of generalization. And CNN reads sentence vectors from both forward and reverse directions to increase the diversity of sentence features. The testing results show that our methods can achieve better performance than many other complex CNN models. In addition, we apply our method in practice of IQA, and the results show the method is effective.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[3]  Zhou Quan,et al.  Combining Statistics-Based and CNN-Based Information for Sentence Classification , 2016, ICTAI.

[4]  Christopher D. Manning,et al.  Fast dropout training , 2013, ICML.

[5]  Dan Roth,et al.  Learning Question Classifiers , 2002, COLING.

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[8]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[9]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[10]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[11]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[12]  Nan Hua,et al.  Universal Sentence Encoder , 2018, ArXiv.

[13]  Cheng Niu,et al.  Location Normalization for Information Extraction , 2002, COLING.

[14]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[15]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[16]  Luísa Coheur,et al.  From symbolic to sub-symbolic information in question classification , 2011, Artificial Intelligence Review.

[17]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[18]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[19]  Christopher D. Manning,et al.  Baselines and Bigrams: Simple, Good Sentiment and Topic Classification , 2012, ACL.

[20]  Wenpeng Yin,et al.  Multichannel Variable-Size Convolution for Sentence Classification , 2015, CoNLL.

[21]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.