Academic News Text Classification Model Based on Attention Mechanism and RCNN

With the expeditious development of Internet technology and academic social media, massive academic news generated by academic social media have provided rich information for scholars to communicate and learn about the latest academic trends. How to effectively classify academic news data and obtain valuable information have become one of the important research directions of information science. Traditional classification methods have the problems of high dimensions, high sparseness and weak feature expression ability, etc. Deep neural network models such as CNN and RNN are also often affected by their own parameters. In this paper we present a deep neural network model based on attention mechanism and RCNN (ARCNN). We capture the context of each word and generate the word vectors with deep bidirectional LSTM layers after preprocessing. Then we use attention mechanism to calculate the attention probability distribution of news titles and contents, effectively highlighting key information. In our experiments, we use news data of academic social network SCHOLAT and Fudan University document classification set to evaluate our model and achieve better results than other widely used text classification algorithms.

[1]  Song Fengxi.  Performance Evaluation Metric for Text Classifiers , 2004 .

[2]  Wen Li,et al.  N-grams based feature selection and text representation for Chinese Text Classification , 2009, Int. J. Comput. Intell. Syst..

[3]  Peng Wang,et al.  Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification , 2016, Neurocomputing.

[4]  Wang Haiyan,et al.  Overview of support vector machine analysis and algorithm , 2014 .

[5]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  D. Karthika Renuka,et al.  SPAM Classification Based on Supervised Learning Using Machine Learning Techniques , 2011 .

[8]  Mevlut Ture,et al.  Comparing performances of logistic regression, classification and regression tree, and neural networks for predicting coronary artery disease , 2008, Expert Syst. Appl..

[9]  Yong Tang,et al.  DNALS: A Recommendation Algorithm Based on Chinese Vocabulary Emotion Analysis of Songs , 2016 .

[10]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[11]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[12]  Geoffrey E. Hinton,et al.  Gated Softmax Classification , 2010, NIPS.

[13]  Heng Zhang,et al.  Improving short text classification by learning vector representations of both words and hidden topics , 2016, Knowl. Based Syst..

[14]  Zhiyuan Liu,et al.  Joint Learning of Character and Word Embeddings , 2015, IJCAI.

[15]  Charu C. Aggarwal,et al.  A Survey of Text Classification Algorithms , 2012, Mining Text Data.