Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification

Hierarchical Text Classification (HTC) is a challenging task that categorizes a textual description within a taxonomic hierarchy. Most of the existing methods focus on modeling the text. Recently, researchers attempt to model the class representations with some resources (e.g., external dictionaries). However, the concept shared among classes which is a kind of domain-specific and fine-grained information has been ignored in previous work. In this paper, we propose a novel concept-based label embedding method that can explicitly represent the concept and model the sharing mechanism among classes for the hierarchical text classification. Experimental results on two widely used datasets prove that the proposed model outperforms several state-of-theart methods. We release our complementary resources (concepts and definitions of classes) for these two datasets to benefit the research on HTC.

[1]  Ting Yao,et al.  Document Gated Reader for Open-Domain Question Answering , 2019, SIGIR.

[2]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[3]  Steffen Staab,et al.  Web Science , 2013, Informatik-Spektrum.

[4]  Lin Xiao,et al.  Hyperbolic Interaction Model For Hierarchical Multi-Label Classification , 2019, AAAI.

[5]  Jason Baldridge,et al.  Hierarchical Discriminative Classification for Text-Based Geolocation , 2014, EMNLP.

[6]  Michael Bain,et al.  B-CNN: Branch Convolutional Neural Network for Hierarchical Classification , 2017, ArXiv.

[7]  Jiawei Han,et al.  Weakly-Supervised Hierarchical Text Classification , 2018, AAAI.

[8]  Marco Antonio Sobrevilla Cabezudo,et al.  Efficient Strategies for Hierarchical Text Classification: External Knowledge and Auxiliary Tasks , 2020, ACL.

[9]  Vasant Honavar,et al.  Learning Classifiers Using Hierarchically Structured Class Taxonomies , 2005, SARA.

[10]  Xuanjing Huang,et al.  Hierarchical Text Classification with Latent Concepts , 2011, ACL.

[11]  Thomas Hofmann,et al.  Hierarchical document categorization with support vector machines , 2004, CIKM '04.

[12]  Marco A. Palomino,et al.  Automatic extraction of keywords for a multi-media search engine using the chi-square test , 2009 .

[13]  Susan T. Dumais,et al.  Hierarchical classification of Web content , 2000, SIGIR '00.

[14]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[15]  Enhong Chen,et al.  Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network Approach , 2019, CIKM.

[16]  Wenhan Xiong,et al.  Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification , 2019, EMNLP.

[17]  Xin Geng,et al.  Hierarchical Classification Based on Label Distribution Learning , 2019, AAAI.

[18]  Brian D. Davison,et al.  Learning Word Embeddings with Chi-Square Weights for Healthcare Tweet Classification , 2017 .

[19]  Fabrizio Sebastiani,et al.  On the Selection of Negative Examples for Hierarchical Text Categorization , 2007 .

[20]  G. A. Barnard.  Introduction to Pearson (1900) On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1992 .

[21]  Alex A. Freitas,et al.  A survey of hierarchical classification across different application domains , 2010, Data Mining and Knowledge Discovery.

[22]  Tian Gan,et al.  Explicit Interaction Model towards Text Classification , 2018, AAAI.

[23]  Yiming Yang,et al.  Recursive regularization for large-scale classification with hierarchical and graphical dependencies , 2013, KDD.

[24]  Chris Biemann,et al.  Hierarchical Multi-label Classification of Text with Capsule Networks , 2019, ACL.

[25]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[26]  Ning Ding,et al.  Hierarchy-Aware Global Model for Hierarchical Text Classification , 2020, ACL.

[27]  Hinrich Schütze,et al.  End-to-End Trainable Attentive Decoder for Hierarchical Entity Classification , 2017, EACL.

[28]  Jianxin Li,et al.  Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN , 2018, WWW.

[29]  Donald E. Brown,et al.  HDLTex: Hierarchical Deep Learning for Text Classification , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[30]  Jae Dong Yang,et al.  Experiment with a Hierarchical Text Categorization Method on WIPO Patent Collections , 2005 .

[31]  Jiawei Han,et al.  Hierarchical Text Classification with Reinforced Label Assignment , 2019, EMNLP.

[32]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[33]  Guoyin Wang,et al.  Joint Embedding of Words and Labels for Text Classification , 2018, ACL.

[34]  Benjamin Van Durme,et al.  Hierarchical Entity Typing via Multi-level Learning to Rank , 2020, ACL.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Yubo Chen,et al.  HyperCore: Hyperbolic and Co-graph Representation for Automatic ICD Coding , 2020, ACL.

[37]  Kostas Tsioutsiouliklis,et al.  Hierarchical Transfer Learning for Multi-label Text Classification , 2019, ACL.

[38]  Philip S. Yu,et al.  Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification , 2019, IEEE Transactions on Knowledge and Data Engineering.

[39]  Fumiyo Fukumoto,et al.  HFT-CNN: Learning Hierarchical Category Structure for Multi-label Short Text Categorization , 2018, EMNLP.

[40]  Rodrigo C. Barros,et al.  Hierarchical Multi-Label Classification Networks , 2018, ICML.