Decomposition-Fusion for Label Distribution Learning

Abstract Label Distribution Learning (LDL) is a general learning framework that assigns an instance to a distribution over a set of labels rather than to a single label or multiple labels. Current LDL methods have proven their effectiveness in many real-life machine learning applications. However, LDL is a generalization of the classification task and as such it is exposed to the same problems as standard classification algorithms, including class-imbalanced, noise, overlapping or irregularities. The purpose of this paper is to mitigate these effects by using decomposition strategies. The technique devised, called Decomposition-Fusion for LDL (DF-LDL), is based on one of the most renowned strategy in decomposition: the One-vs-One scheme, which we adapt to be able to deal with LDL datasets. In addition, we propose a competent fusion method that allows us to discard non-competent classifiers when their output is probably not of interest. The effectiveness of the proposed DF-LDL method is verified on several real-world LDL datasets on which we have carried out two types of experiments. First, comparing our proposal with the base learners and, second, comparing our proposal with the state-of-the-art LDL algorithms. DF-LDL shows significant improvements in both experiments.

[1]  Marco Zaffalon,et al.  Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis , 2016, J. Mach. Learn. Res..

[2]  Jianhua Dai,et al.  Label Distribution Feature Selection Based on Mutual Information in Fuzzy Rough Set Theory , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[3]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[4]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[5]  Grigorios Tsoumakas,et al.  MULAN: A Java Library for Multi-Label Learning , 2011, J. Mach. Learn. Res..

[6]  Lior Rokach,et al.  Data Mining with Decision Trees - Theory and Applications , 2007, Series in Machine Perception and Artificial Intelligence.

[7]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[8]  Francisco Herrera,et al.  Dynamic classifier selection for One-vs-One strategy: Avoiding non-competent classifiers , 2013, Pattern Recognit..

[9]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[10]  Yang Yu,et al.  Integration of an improved dynamic ensemble selection approach to enhance one-vs-one scheme , 2018, Eng. Appl. Artif. Intell..

[11]  Wenyu Liu,et al.  Structured random forest for label distribution learning , 2018, Neurocomputing.

[12]  José Ramón Cano,et al.  ProLSFEO-LDL: Prototype Selection and Label- Specific Feature Evolutionary Optimization for Label Distribution Learning , 2020, Applied Sciences.

[13]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[14]  Gustavo E. A. P. A. Batista,et al.  Class Imbalances versus Class Overlapping: An Analysis of a Learning System Behavior , 2004, MICAI.

[15]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[16]  Krzysztof J. Cios,et al.  Review of ensembles of multi-label classifiers: Models, experimental study and prospects , 2018, Inf. Fusion.

[17]  Ke Wang,et al.  Binary Coding based Label Distribution Learning , 2018, IJCAI.

[18]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[19]  Kai Zhao,et al.  Label Distribution Learning Forests , 2017, NIPS.

[20]  Jufeng Yang,et al.  Joint Image Emotion Classification and Distribution Learning via Deep Convolutional Neural Network , 2017, IJCAI.

[21]  D C CavalcantiGeorge,et al.  Dynamic classifier selection , 2018 .

[22]  Xiao Sun,et al.  Discriminate the Falsely Predicted Protein-Coding Genes in Aeropyrum Pernix K1 Genome Based on Graphical Representation , 2012 .

[23]  Sung-Hyuk Cha Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions , 2007 .

[24]  Francisco Herrera,et al.  Emerging topics and challenges of learning from noisy data in nonstandard classification: a survey beyond binary class noise , 2018, Knowledge and Information Systems.

[25]  Francisco Herrera,et al.  Using the One-vs-One decomposition to improve the performance of class noise filters via an aggregation strategy in multi-class classification problems , 2015, Knowl. Based Syst..

[26]  María José del Jesús,et al.  KEEL 3.0: An Open Source Software for Multi-Stage Analysis in Data Mining , 2017, Int. J. Comput. Intell. Syst..

[27]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Zhi-Hua Zhou,et al.  Facial Age Estimation by Learning from Label Distributions , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Francisco Charte,et al.  Multilabel Classification , 2016, Springer International Publishing.

[31]  Francisco Herrera,et al.  rNPBST: An R Package Covering Non-parametric and Bayesian Statistical Tests , 2017, HAIS.

[32]  Bianca Zadrozny,et al.  Transforming classifier scores into accurate multiclass probability estimates , 2002, KDD.

[33]  Francisco Herrera,et al.  An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes , 2011, Pattern Recognit..

[34]  Jingying Chen,et al.  Head pose estimation using improved label distribution learning with fewer annotations , 2019, Multimedia Tools and Applications.

[35]  Xiuyi Jia,et al.  Label Distribution Learning by Exploiting Sample Correlations Locally , 2018, AAAI.

[36]  Chongsheng Zhang,et al.  An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme , 2018, Knowl. Based Syst..

[37]  Francisco Herrera,et al.  Analyzing the presence of noise in multi-class problems: alleviating its influence with the One-vs-One decomposition , 2012, Knowledge and Information Systems.

[38]  Robert E. Schapire,et al.  Hierarchical multi-label prediction of gene function , 2006, Bioinform..

[39]  Jianxin Wu,et al.  Deep Label Distribution Learning With Label Ambiguity , 2016, IEEE Transactions on Image Processing.

[40]  Xin Geng,et al.  Discrete Binary Coding based Label Distribution Learning , 2019, IJCAI.

[41]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[42]  Francisco Herrera,et al.  Empowering difficult classes with a similarity-based aggregation in multi-class classification problems , 2014, Inf. Sci..

[43]  Xin Geng,et al.  Crowd counting in public video surveillance by label distribution learning , 2015, Neurocomputing.

[44]  Emanuel Aldea,et al.  Evidential framework for Error Correcting Output Code classification , 2018, Eng. Appl. Artif. Intell..

[45]  Mikel Galar,et al.  Addressing the Overlapping Data Problem in Classification Using the One-vs-One Decomposition Strategy , 2019, IEEE Access.

[46]  Eyke Hüllermeier,et al.  Binary Decomposition Methods for Multipartite Ranking , 2009, ECML/PKDD.

[47]  Francisco Herrera,et al.  Learning from Imbalanced Data Sets , 2018, Springer International Publishing.

[48]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[49]  Jing Wang,et al.  Classification with Label Distribution Learning , 2019, IJCAI.

[50]  George D. C. Cavalcanti,et al.  Dynamic classifier selection: Recent advances and perspectives , 2018, Inf. Fusion.

[51]  Yi Ren,et al.  Sense Beauty by Label Distribution Learning , 2017, IJCAI.

[52]  Hong Shi,et al.  Label Distribution Learning Based on Ensemble Neural Networks , 2018, ICONIP.

[53]  Liang Gao,et al.  Personality Recognition on Social Media With Label Distribution Learning , 2017, IEEE Access.

[54]  Celine Vens,et al.  Labelling strategies for hierarchical multi-label classification techniques , 2016, Pattern Recognit..

[55]  Bidyut Baran Chaudhuri,et al.  Handling data irregularities in classification: Foundations, trends, and future challenges , 2018, Pattern Recognit..

[56]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[57]  Xin Geng,et al.  Pre-release Prediction of Crowd Opinion on Movies by Label Distribution Learning , 2015, IJCAI.

[58]  Francisco Herrera,et al.  NMC: nearest matrix classification - A new combination model for pruning One-vs-One ensembles by transforming the aggregation problem , 2017, Inf. Fusion.

[59]  D. Kibler,et al.  Instance-based learning algorithms , 2004, Machine Learning.

[60]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.