Multi-Dimensional Classification via Sparse Label Encoding

In multi-dimensional classification (MDC), there are multiple class variables in the output space with each of them corresponding to one heterogeneous class space. Due to the heterogeneity of class spaces, it is quite challenging to consider the dependencies among class variables when learning from MDC examples. In this paper, we propose a novel MDC approach named SLEM which learns the predictive model in an encoded label space instead of the original heterogeneous one. Specifically, SLEM works in an encodingtraining-decoding framework. In the encoding phase, each class vector is mapped into a realvalued one via three cascaded operations including pairwise grouping, one-hot conversion and sparse linear encoding. In the training phase, a multi-output regression model is learned within the encoded label space. In the decoding phase, the predicted class vector is obtained by adapting orthogonal matching pursuit over outputs of the learned multi-output regression model. Experimental results clearly validate the superiority of SLEM against state-of-the-art MDC approaches.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[3]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[4]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[5]  Luca Martino,et al.  Efficient monte carlo methods for multi-dimensional learning with classifier chains , 2012, Pattern Recognit..

[6]  Concha Bielza,et al.  A survey on multi‐output regression , 2015, WIREs Data Mining Knowl. Discov..

[7]  José Antonio Lozano,et al.  Using Multidimensional Bayesian Network Classifiers to Assist the Treatment of Multiple Sclerosis , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Min-Ling Zhang,et al.  Multi-Dimensional Classification via kNN Feature Augmentation , 2019, AAAI.

[9]  Min-Ling Zhang,et al.  Multi-dimensional classification via stacked dependency exploitation , 2020, Science China Information Sciences.

[10]  Surendra P. Verma,et al.  A statistically coherent robust multidimensional classification scheme for water. , 2021, The Science of the total environment.

[11]  Hsuan-Tien Lin,et al.  Multilabel Classification with Principal Label Space Transformation , 2012, Neural Computation.

[12]  Chen Chen,et al.  Incorporating Label Embedding and Feature Augmentation for Multi-Dimensional Classification , 2020, AAAI.

[13]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[14]  Concha Bielza,et al.  Multi-dimensional classification with Bayesian networks , 2011, Int. J. Approx. Reason..

[15]  Marc Teboulle,et al.  A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications , 2017, Math. Oper. Res..

[16]  Weiwei Liu,et al.  An Easy-to-hard Learning Paradigm for Multiple Classes and Multiple Labels , 2017, J. Mach. Learn. Res..

[17]  Janneke H. Bolt,et al.  Balanced sensitivity functions for tuning multi-dimensional Bayesian network classifiers , 2017, Int. J. Approx. Reason..

[18]  Concha Bielza,et al.  Bayesian Chain Classifiers for Multidimensional Classification , 2011, IJCAI.

[19]  Concha Bielza,et al.  Multi-dimensional Bayesian network classifiers: A survey , 2020, Artificial Intelligence Review.

[20]  Hagit Shatkay,et al.  Multi-dimensional classification of biomedical text: Toward automated, practical provision of high-utility text to diverse users , 2008, Bioinform..

[21]  Weiwei Liu,et al.  Compact Multi-Label Learning , 2018, AAAI.

[22]  Songcan Chen,et al.  Multi-dimensional classification via a metric approach , 2018, Neurocomputing.

[23]  Concha Bielza,et al.  International Journal of Approximate Reasoning Tractability of most probable explanations in multidimensional Bayesian network classifiers ✩ , 2022 .

[24]  Concha Bielza,et al.  Multi-Dimensional Classification with Super-Classes , 2014, IEEE Transactions on Knowledge and Data Engineering.

[25]  Sanyang Liu,et al.  A hybrid method for learning multi-dimensional Bayesian network classifiers based on an optimization model , 2015, Applied Intelligence.

[26]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[27]  Weiwei Liu,et al.  Sparse Extreme Multi-label Learning with Oracle Property , 2019, ICML.

[28]  Hiroaki Harai,et al.  Multi-Target Classification Based Automatic Virtual Resource Allocation Scheme , 2019, IEICE Trans. Inf. Syst..

[29]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[30]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[31]  Weiwei Liu,et al.  Metric Learning for Multi-Output Tasks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.