Active learning for hierarchical multi-label classification

Due to technological advances, a massive amount of data is produced daily, presenting challenges for application areas where data needs to be labelled by a domain specialist or by expensive procedures, in order to be useful for supervised machine learning purposes. In order to select which data points will provide more information when labelled, one can make use of active learning methods. Active learning (AL) is a subfield of machine learning which addresses methods to build models with fewer, but more representative instances. Even though AL has been vastly studied, it has not been thoroughly investigated in hierarchical multi-label classification, a learning task where multiple class labels can be assigned to an instance and these labels are hierarchically structured. In this work, we provide a public framework containing baseline and state-of-the-art algorithms suitable for this task. Additionally, we also propose a new algorithm, namely Hierarchical Query-By-Committee (H-QBC), which is validated on datasets from different domains. Our results show that H-QBC is capable of providing superior predictive performance results compared to its competitors, while being computationally efficient and parameter free.

[1]  Pengpeng Zhao,et al.  Multi-label active learning for image classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[2]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Inducing Hierarchical Multi-label Classification rules with Genetic Algorithms , 2019, Appl. Soft Comput..

[3]  Saso Dzeroski,et al.  Predicting gene function using hierarchical multi-label decision tree ensembles , 2010, BMC Bioinformatics.

[4]  Sheng-Jun Huang,et al.  Cost-Effective Active Learning for Hierarchical Multi-Label Classification , 2018, IJCAI.

[5]  Zheng Chen,et al.  Effective multi-label active learning for text classification , 2009, KDD.

[6]  Erik Strumbelj,et al.  Explaining prediction models and individual predictions with feature contributions , 2014, Knowledge and Information Systems.

[7]  Zhi-Hua Zhou,et al.  Active Query Driven by Uncertainty and Diversity for Incremental Multi-label Learning , 2013, 2013 IEEE 13th International Conference on Data Mining.

[8]  Sebastián Ventura,et al.  Effective active learning strategy for multi-label learning , 2018, Neurocomputing.

[9]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[10]  Yiming Yang,et al.  RCV1: A New Benchmark Collection for Text Categorization Research , 2004, J. Mach. Learn. Res..

[11]  Concha Bielza,et al.  A survey on multi‐output regression , 2015, WIREs Data Mining Knowl. Discov..

[12]  Pengpeng Zhao,et al.  Multi-Label Active Learning with Chi-Square Statistics for Image Classification , 2015, ICMR.

[13]  Doug Downey,et al.  Learning Hierarchically Decomposable Concepts with Active Over-Labeling , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[14]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[15]  Rong Jin,et al.  Batch mode active learning and its application to medical image classification , 2006, ICML.

[16]  Michelangelo Ceci,et al.  Semi-supervised trees for multi-target regression , 2018, Inf. Sci..

[17]  Jesse Davis,et al.  Learning from positive and unlabeled data: a survey , 2018, Machine Learning.

[18]  Zixiang Wang,et al.  Ontological function annotation of long non‐coding RNAs through hierarchical multi‐label classification , 2018, Bioinform..

[19]  Min-Ling Zhang,et al.  Ml-rbf: RBF Neural Networks for Multi-Label Learning , 2009, Neural Processing Letters.

[20]  Pengpeng Zhao,et al.  Multi-label active learning with low-rank mapping for image classification , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[21]  Sethuraman Panchanathan,et al.  BatchRank: A Novel Batch Mode Active Learning Framework for Hierarchical Classification , 2015, KDD.

[22]  Rob J. Hyndman,et al.  Hierarchical Forecasting , 2019 .

[23]  Xian-Sheng Hua,et al.  Two-Dimensional Active Learning for image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jennifer G. Dy,et al.  Active Learning from Crowds , 2011, ICML.

[25]  Tao Jiang,et al.  Marine Wireless Big Data: Efficient Transmission, Related Applications, and Challenges , 2017, IEEE Wireless Communications.

[26]  Saso Dzeroski,et al.  Tree ensembles for predicting structured outputs , 2013, Pattern Recognit..

[27]  Pengpeng Zhao,et al.  An Active Learning Approach for Multi-Label Image Classification with Sample Noise , 2018, Int. J. Pattern Recognit. Artif. Intell..

[28]  Xin Li,et al.  Active Learning with Multi-Label SVM Classification , 2013, IJCAI.

[29]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Hierarchical classification of Gene Ontology-based protein functions with neural networks , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[30]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Reduction strategies for hierarchical multi-label classification in protein function prediction , 2016, BMC Bioinformatics.

[31]  Fei Wang,et al.  Batch Mode Active Learning with Hierarchical-Structured Embedded Variance , 2014, SDM.

[32]  Saso Dzeroski,et al.  Decision trees for hierarchical multi-label classification , 2008, Machine Learning.

[33]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  A genetic algorithm for Hierarchical Multi-Label Classification , 2012, SAC '12.

[34]  Pengpeng Zhao,et al.  Active Multi-label Learning with Optimal Label Subset Selection , 2014, ADMA.

[35]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[36]  LarrañagaPedro,et al.  A survey on multi-output regression , 2015 .

[37]  Pengpeng Zhao,et al.  Active learning with label correlation exploration for multi-label image classification , 2017, IET Comput. Vis..

[38]  Giorgio Valentini,et al.  True Path Rule Hierarchical Ensembles for Genome-Wide Gene Function Prediction , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[39]  L MinkuLeandro,et al.  Ensemble learning for data stream analysis , 2017 .

[40]  Guangyuan Fu,et al.  NewGOA: Predicting New GO Annotations of Proteins by Bi-Random Walks on a Hybrid Graph , 2018, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[41]  Lei Wang,et al.  Multilabel SVM active learning for image classification , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[42]  Yiming Yang,et al.  The Enron Corpus: A New Dataset for Email Classi(cid:12)cation Research , 2004 .

[43]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[44]  Ashish Kapoor,et al.  Active learning for sparse bayesian multilabel classification , 2014, KDD.

[45]  Celine Vens,et al.  Machine learning for discovering missing or wrong protein function annotations , 2019, BMC Bioinformatics.

[46]  Pengpeng Zhao,et al.  Multi-label active learning with label correlation for image classification , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[47]  Sethuraman Panchanathan,et al.  Optimal batch selection for active learning in multi-label classification , 2011, ACM Multimedia.

[48]  Hsuan-Tien Lin,et al.  Multi-label Active Learning with Auxiliary Learner , 2011, ACML.

[49]  Alok N. Choudhary,et al.  On active learning in hierarchical classification , 2012, CIKM '12.

[50]  Yang Yu,et al.  Subset Selection by Pareto Optimization , 2015, NIPS.

[51]  Kai Yang,et al.  Active Learning for Wireless IoT Intrusion Detection , 2018, IEEE Wireless Communications.

[52]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[53]  Giuseppe De Pietro,et al.  Deep neural network for hierarchical extreme multi-label text classification , 2019, Appl. Soft Comput..

[54]  Xiao Li,et al.  Active Learning for Hierarchical Text Classification , 2012, PAKDD.

[55]  Gisele L. Pappa,et al.  Top-down strategies for hierarchical classification of transposable elements with neural networks , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[56]  Larisa Shwartz,et al.  Knowledge Guided Hierarchical Multi-Label Classification Over Ticket Data , 2017, IEEE Transactions on Network and Service Management.

[57]  Konstantinos Pliakos,et al.  Mining features for biomedical data using clustering tree ensembles , 2018, J. Biomed. Informatics.

[58]  Saso Dzeroski,et al.  Predictive Clustering Trees for Hierarchical Multi-Target Regression , 2017, IDA.

[59]  João Gama,et al.  Ensemble learning for data stream analysis: A survey , 2017, Inf. Fusion.

[60]  Holger H. Hoos,et al.  A survey on semi-supervised learning , 2019, Machine Learning.

[61]  James D. Duin Hierarchical Active Learning Application to Mitochondrial Disease Protein Dataset , 2017 .

[62]  Yannis Papanikolaou,et al.  Multi-label active learning: key issues and a novel query strategy , 2017, Evolving Systems.

[63]  Rong Jin,et al.  Active Learning by Querying Informative and Representative Examples , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Yang Wang,et al.  Multilabel Image Classification Via High-Order Label Correlation Driven Active Learning , 2014, IEEE Transactions on Image Processing.

[65]  Michelangelo Ceci,et al.  Self-training for multi-target regression with tree ensembles , 2017, Knowl. Based Syst..

[66]  Xiangliang Zhang,et al.  A Literature Review of Gene Function Prediction by Modeling Gene Ontology , 2020, Frontiers in Genetics.

[67]  Rasoul Karimi,et al.  Active Learning for Recommender Systems , 2015, KI - Künstliche Intelligenz.

[68]  Klaus Brinker,et al.  On Active Learning in Multi-label Classification , 2005, GfKl.

[69]  Xiao Li,et al.  Effective Top-Down Active Learning for Hierarchical Text Classification , 2013, PAKDD.

[70]  Rodrigo C. Barros,et al.  Hierarchical Multi-Label Classification Networks , 2018, ICML.

[71]  Yang Wang,et al.  Batch mode active learning for multi-label image classification with informative label correlation mining , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).