Text Classification using Triplet Capsule Networks

Most existing methods only consider the local features of the samples, and their experimental results show better performance than traditional Non-deep learning methods. However, in these methods, the global features of the sample space are usually ignored, and these ignored global features will affect the classification accuracy. To solve this problem, a novel triple capsule network framework is proposed to text classification. The training in the first stage, to obtain a basic capsule network for obtaining local features. Then, three capsule networks sharing parameters are combined spatially, and the triplet loss function is used in the second stage of training. By comparative learning, the capsule network can learn global features that can represent the spatial distance between different categories. Through comparison experiments on six datasets and ten general benchmark algorithms, the results show that our results is the first in the four datasets.

[1]  Jia Wu,et al.  Artificial immune system for attribute weighted Naive Bayes classification , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[2]  Bruno Trstenjak,et al.  on Intelligent Manufacturing and Automation , 2013 KNN with TF-IDF Based Framework for Text Categorization , 2014 .

[3]  Noah A. Smith,et al.  Variational Pretraining for Semi-supervised Text Classification , 2019, ACL.

[4]  Chuan Zhou,et al.  Big social network influence maximization via recursively estimating influence spread , 2016, Knowl. Based Syst..

[5]  Ting Liu,et al.  Learning Semantic Representations of Users and Products for Document Level Sentiment Classification , 2015, ACL.

[6]  Huang Zou,et al.  Sentiment Classification Using Machine Learning Techniques with Syntax Features , 2015, 2015 International Conference on Computational Science and Computational Intelligence (CSCI).

[7]  Hui Xiong,et al.  SAIL: Summation-bAsed Incremental Learning for Information-Theoretic Text Clustering , 2013, IEEE Transactions on Cybernetics.

[8]  Ion Androutsopoulos,et al.  Large-Scale Multi-Label Text Classification on EU Legislation , 2019, ACL.

[9]  Ulas Bagci,et al.  Capsules for Object Segmentation , 2018, ArXiv.

[10]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[11]  Min Yang,et al.  Investigating Capsule Networks with Dynamic Routing for Text Classification , 2018, EMNLP.

[12]  Premkumar Natarajan,et al.  CapsuleGAN: Generative Adversarial Capsule Network , 2018, ECCV Workshops.

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Yang Jin,et al.  Capsule Network Performance on Complex Data , 2017, ArXiv.

[15]  Zhiyuan Liu,et al.  Neural Relation Extraction with Selective Attention over Instances , 2016, ACL.

[16]  Tong Zhang,et al.  Deep Pyramid Convolutional Neural Networks for Text Categorization , 2017, ACL.

[17]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[18]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[19]  Baoxin Wang,et al.  Disconnected Recurrent Neural Networks for Text Categorization , 2018, ACL.

[20]  Jing Li,et al.  Topic Memory Networks for Short Text Classification , 2018, EMNLP.

[21]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[22]  Xiaoyan Zhu,et al.  Sentiment Analysis by Capsules , 2018, WWW.

[23]  Yaohui Jin,et al.  MCapsNet: Capsule Network for Text with Multi-Task Learning , 2018, EMNLP.

[24]  Xuanjing Huang,et al.  Recurrent Neural Network for Text Classification with Multi-Task Learning , 2016, IJCAI.

[25]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[26]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[27]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[28]  Min Zhu,et al.  Text Classification Using Gated and Transposed Attention Networks , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[29]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[30]  Andrew M. Dai,et al.  Adversarial Training Methods for Semi-Supervised Text Classification , 2016, ICLR.

[31]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[32]  William Yang Wang,et al.  Towards Explainable NLP: A Generative Explanation Framework for Text Classification , 2018, ACL.

[33]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[34]  Peng Zhou,et al.  Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling , 2016, COLING.

[35]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[36]  Kathleen M. Carley,et al.  Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks , 2018, SBP-BRiMS.

[37]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[38]  Zhi-Li Zhang,et al.  Graph Capsule Convolutional Neural Networks , 2018, ArXiv.

[39]  Geoffrey E. Hinton,et al.  Matrix capsules with EM routing , 2018, ICLR.

[40]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[41]  Jie Cao,et al.  An Improved Protocol for Deadlock and Livelock Avoidance Resource Co-allocation in Network Computing , 2010, World Wide Web.

[42]  Xindong Wu,et al.  Multi-Instance Learning with Discriminative Bag Mapping , 2018, IEEE Transactions on Knowledge and Data Engineering.

[43]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[44]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[45]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[46]  Min Yang,et al.  Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications , 2019, ACL.

[47]  Guoyin Wang,et al.  Joint Embedding of Words and Labels for Text Classification , 2018, ACL.

[48]  Chengqi Zhang,et al.  Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling , 2018, ICLR.

[49]  Nigel Collier,et al.  Sentiment Analysis using Support Vector Machines with Diverse Information Sources , 2004, EMNLP.

[50]  Nan Hua,et al.  Universal Sentence Encoder for English , 2018, EMNLP.

[51]  Hongbo Xu,et al.  Adapting Naive Bayes to Domain Adaptation for Sentiment Analysis , 2009, ECIR.

[52]  Tieyun Qian,et al.  Transfer Capsule Network for Aspect Level Sentiment Classification , 2019, ACL.

[53]  Wei Zhao,et al.  Investigating the transferring capability of capsule networks for text classification , 2019, Neural Networks.

[54]  Chuan Zhou,et al.  Collaborative Dynamic Sparse Topic Regression with User Profile Evolution for Item Recommendation , 2017, AAAI.

[55]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[56]  Xuanjing Huang,et al.  Information Aggregation via Dynamic Routing for Sequence Encoding , 2018, COLING.

[57]  Yunming Ye,et al.  Cross-Domain Sentiment Classification by Capsule Network With Semantic Rules , 2018, IEEE Access.

[58]  Tong Zhang,et al.  Effective Use of Word Order for Text Categorization with Convolutional Neural Networks , 2014, NAACL.