Multi-layer Convolutional Neural Network Model Based on Prior Knowledge of Knowledge Graph for Text Classification

Deep neural networks can obtain effective hierarchical representations, which make them perform well in image recognition and speech recognition. However, the disadvantages of deep learning relying on large-scale annotation data also limit its development. To this end, a multi-layer convolutional neural network based on knowledge graph prior knowledge is proposed in this paper, which acquires text features from finer granularity, and using the existing prior knowledge which increases granularity information to enhance the semantic information of the text, and to reduce the dependence of the model on large-scale samples. At the same time, it can reduce the over-fitting problem of the model, and improve classification accuracy. This model performs well in several large data sets. By introducing prior knowledge to TextCNN, a classical deep neural model, it has also verified the validity of using prior knowledge through experiments.

[1]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[2]  Ming-Wei Chang,et al.  Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base , 2015, ACL.

[3]  Yaohui Jin,et al.  Multi-Task Label Embedding for Text Classification , 2017, EMNLP.

[4]  Giuseppe Ottaviano,et al.  Fast and Space-Efficient Entity Linking for Queries , 2015, WSDM.

[5]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[6]  Andrei Popescu-Belis,et al.  Multilingual Hierarchical Attention Networks for Document Classification , 2017, IJCNLP.

[7]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[8]  Jun Wang,et al.  Learning text representation using recurrent convolutional neural network with highway layers , 2016, SIGIR 2016.

[9]  Roi Blanco,et al.  Lightweight Multilingual Entity Extraction and Linking , 2017, WSDM.

[10]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[11]  Xuanjing Huang,et al.  Recurrent Neural Network for Text Classification with Multi-Task Learning , 2016, IJCAI.

[12]  Efstathios Stamatatos,et al.  Words versus Character n-Grams for Anti-Spam Filtering , 2007, Int. J. Artif. Intell. Tools.

[13]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[14]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[15]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[18]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Krzysztof Marasek,et al.  Neural-based machine translation for medical text domain. Based on European Medicines Agency leaflet texts , 2015, CENTERIS/ProjMAN/HCist.

[21]  Peng Wang,et al.  Semantic Clustering and Convolutional Neural Network for Short Text Categorization , 2015, ACL.

[22]  Ralph Grishman,et al.  Relation Extraction: Perspective from Convolutional Neural Networks , 2015, VS@HLT-NAACL.

[23]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[24]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[25]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.