Label distribution feature selection for multi-label classification with rough set

Abstract Multi-label learning deals with cases where every instance corresponds to multiple labels. The objective is to learn mapping from an instance to a relevant label set. Existing multi-label learning approaches assume that the significance for all related labels is same for every instance. Several problems of label ambiguity can be dealt with using multi-label learning, but some practical applications with significance among related labels for every instance cannot be effectively processed. To achieve superior results by conducting different significance of labels, label distribution learning is used for such applications. First, the probability model and rough set are embedded in the labeling significance, thus more supervised information can be obtained from original multi-label data. Subsequently, to resolve the feature selection problem of label distribution data, according to the feature dependency and the rough set, a novel feature selection algorithm for multi-label classification is designed. Finally, to verify the effectiveness of the proposed algorithms, an extensive experiment is conducted on 15 real-world multiple label data sets. The performance of the proposed algorithm through the multi-label classifier is compared with seven state-of-the-art approaches, thereby indicating the applicability and effectiveness of label distribution feature selection.

[1]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[2]  Zhiwen Yu,et al.  Multitask Feature Selection by Graph-Clustered Feature Sharing , 2020, IEEE Transactions on Cybernetics.

[3]  Witold Pedrycz,et al.  Positive approximation: An accelerator for attribute reduction in rough set theory , 2010, Artif. Intell..

[4]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Global and Local Label Correlation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[5]  Ning Xu,et al.  Label Enhancement for Label Distribution Learning , 2018, IEEE Transactions on Knowledge and Data Engineering.

[6]  Cuiping Li,et al.  Discernibility matrix based incremental feature selection on fused decision tables , 2020, Int. J. Approx. Reason..

[7]  Hua Li,et al.  A novel attribute reduction approach for multi-label data based on rough set theory , 2016, Inf. Sci..

[8]  Zhi-Hua Zhou,et al.  Multi-instance multi-label learning , 2008, Artif. Intell..

[9]  David Zhang,et al.  Multi-Label Dictionary Learning for Image Annotation , 2016, IEEE Transactions on Image Processing.

[10]  Wei-Zhi Wu,et al.  Maximal-Discernibility-Pair-Based Approach to Attribute Reduction in Fuzzy Rough Sets , 2018, IEEE Transactions on Fuzzy Systems.

[11]  Yiyu Yao,et al.  Three-way decisions with probabilistic rough sets , 2010, Inf. Sci..

[12]  Jing-Yu Yang,et al.  Multi-label learning with label-specific feature reduction , 2016, Knowl. Based Syst..

[13]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[14]  Jieping Ye,et al.  Extracting shared subspace for multi-label classification , 2008, KDD.

[15]  Feiping Nie,et al.  SVM based multi-label learning with missing labels for image annotation , 2018, Pattern Recognit..

[16]  Xin Geng,et al.  Emotion Distribution Recognition from Facial Expressions , 2015, ACM Multimedia.

[17]  Sebastian Widz,et al.  Decision bireducts and decision reducts - a comparison , 2017, Int. J. Approx. Reason..

[18]  Hossein Nezamabadi-pour,et al.  Multilabel feature selection: A comprehensive review and guiding experiments , 2018, WIREs Data Mining Knowl. Discov..

[19]  Zhi-Hua Zhou,et al.  Multilabel dimensionality reduction via dependence maximization , 2008, TKDD.

[20]  Hossein Nezamabadi-pour,et al.  A label-specific multi-label feature selection algorithm based on the Pareto dominance concept , 2019, Pattern Recognit..

[21]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[22]  Yuwen Li,et al.  Attribute reduction for multi-label learning with fuzzy rough set , 2018, Knowl. Based Syst..

[23]  Shunxiang Wu,et al.  Online multi-label streaming feature selection based on neighborhood rough set , 2018, Pattern Recognit..

[24]  Xin Geng,et al.  Leveraging Implicit Relative Labeling-Importance Information for Effective Multi-label Learning , 2015, 2015 IEEE International Conference on Data Mining.

[25]  Yu-Chi Chen,et al.  The optimal combination of feature selection and data discretization: An empirical study , 2019, Inf. Sci..

[26]  Yiyu Yao,et al.  Three-way decision and granular computing , 2018, Int. J. Approx. Reason..

[27]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[28]  Yu-Chiang Frank Wang,et al.  Learning Deep Latent Spaces for Multi-Label Classification , 2017, ArXiv.

[29]  Qinghua Hu,et al.  Multi-label feature selection with missing labels , 2018, Pattern Recognit..

[30]  William Zhu,et al.  Sparse Graph Embedding Unsupervised Feature Selection , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Duoqian Miao,et al.  Three-way attribute reducts , 2017, Int. J. Approx. Reason..

[32]  Yuhua Qian,et al.  Accelerator for supervised neighborhood based attribute reduction , 2020, Int. J. Approx. Reason..

[33]  Geoff Holmes,et al.  Multi-label Classification Using Ensembles of Pruned Sets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[34]  Zhi-Hua Zhou,et al.  On the Consistency of Multi-Label Learning , 2011, COLT.

[35]  Xian-Sheng Hua,et al.  A transductive multi-label learning approach for video concept detection , 2011, Pattern Recognit..

[36]  Dae-Won Kim,et al.  Optimization approach for feature selection in multi-label classification , 2017, Pattern Recognit. Lett..

[37]  Tommy W. S. Chow,et al.  Robust non-negative sparse graph for semi-supervised multi-label learning with missing labels , 2018, Inf. Sci..

[38]  Liang-qun Li,et al.  A novel T-S fuzzy particle filtering algorithm based on fuzzy C-regression clustering , 2020, Int. J. Approx. Reason..

[39]  Dominik Slezak,et al.  On resilient feature selection: Computational foundations of r-C-reducts , 2019, Inf. Sci..

[40]  Witold Pedrycz,et al.  Multiple Relevant Feature Ensemble Selection Based on Multilayer Co-Evolutionary Consensus MapReduce , 2020, IEEE Transactions on Cybernetics.

[41]  Qinghua Hu,et al.  Neighborhood classifiers , 2008, Expert Syst. Appl..

[42]  Rung Ching Chen,et al.  Semi-supervised multi-label feature selection via label correlation analysis with l1-norm graph embedding , 2017, Image Vis. Comput..

[43]  Shunxiang Wu,et al.  Feature selection for multi-label learning based on kernelized fuzzy rough sets , 2018, Neurocomputing.

[44]  Hsuan-Tien Lin,et al.  Cost-sensitive label embedding for multi-label classification , 2017, Machine Learning.

[45]  Guoxian Yu,et al.  Feature selection with missing labels based on label compression and local feature correlation , 2020, Neurocomputing.

[46]  Jianfeng Xu,et al.  A three-way selective ensemble model for multi-label classification , 2018, Int. J. Approx. Reason..

[47]  Shou-De Lin,et al.  Cost-Sensitive Multi-Label Learning for Audio Tag Annotation and Retrieval , 2011, IEEE Transactions on Multimedia.

[48]  Jianghong Ma,et al.  Label-specific feature selection and two-level label recovery for multi-label classification with missing labels , 2019, Neural Networks.

[49]  Dae-Won Kim,et al.  SCLS: Multi-label feature selection based on scalable criterion for large label set , 2017, Pattern Recognit..

[50]  Witold Pedrycz,et al.  Label distribution learning: A local collaborative mechanism , 2020, Int. J. Approx. Reason..

[51]  Dae-Won Kim,et al.  Fast multi-label feature selection based on information-theoretic feature ranking , 2015, Pattern Recognit..

[52]  Bo Wang,et al.  Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification , 2013, 2013 IEEE International Conference on Computer Vision.

[53]  Sebastián Ventura,et al.  Scalable extensions of the ReliefF algorithm for weighting and selecting features on the multi-label learning context , 2015, Neurocomputing.

[54]  Alexandre Bernardino,et al.  Matrix Completion for Multi-label Image Classification , 2011, NIPS.

[55]  Deyu Li,et al.  Knowledge structure preserving fuzzy attribute reduction in fuzzy formal context , 2019, Int. J. Approx. Reason..

[56]  Qinghua Hu,et al.  Streaming Feature Selection for Multilabel Learning Based on Fuzzy Mutual Information , 2017, IEEE Transactions on Fuzzy Systems.

[57]  Yonghong Xie,et al.  Incremental feature selection for dynamic hybrid data using neighborhood rough set , 2020, Knowl. Based Syst..

[58]  Jianhua Dai,et al.  Feature selection via normative fuzzy information weight with application into tumor classification , 2020, Appl. Soft Comput..

[59]  Craig A. Knoblock,et al.  A Survey of Digital Map Processing Techniques , 2014, ACM Comput. Surv..

[60]  Shunxiang Wu,et al.  Online Multi-label Group Feature Selection , 2017, Knowl. Based Syst..

[61]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[62]  Jie Duan,et al.  Multi-label feature selection based on neighborhood mutual information , 2016, Appl. Soft Comput..

[63]  Jiye Liang,et al.  Ieee Transactions on Knowledge and Data Engineering 1 a Group Incremental Approach to Feature Selection Applying Rough Set Technique , 2022 .

[64]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[65]  Xin Geng,et al.  Pre-release Prediction of Crowd Opinion on Movies by Label Distribution Learning , 2015, IJCAI.

[66]  Sebastián Ventura,et al.  Distributed nearest neighbor classification for large-scale multi-label data on spark , 2018, Future Gener. Comput. Syst..

[67]  Caihui Liu,et al.  Sequential three-way decisions via multi-granularity , 2020, Inf. Sci..

[68]  Jianhua Dai,et al.  Label Distribution Feature Selection Based on Mutual Information in Fuzzy Rough Set Theory , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[69]  Ju-Sheng Mi,et al.  Optimal granulation selection for multi-label data based on multi-granulation rough sets , 2018, Granular Computing.

[70]  Zhi-Hua Zhou,et al.  Multi-Label Learning by Exploiting Label Correlations Locally , 2012, AAAI.

[71]  Stefan Kramer,et al.  A label compression method for online multi-label classification , 2018, Pattern Recognit. Lett..

[72]  Yang Gao,et al.  Joint multi-label classification and label correlations with missing labels and feature selection , 2019, Knowl. Based Syst..

[73]  Xiao Zhang,et al.  Feature selection in mixed data: A method using a novel fuzzy rough set-based information entropy , 2016, Pattern Recognit..

[74]  Dae-Won Kim,et al.  MFC: Initialization method for multi-label feature selection based on conditional mutual information , 2020, Neurocomputing.

[75]  Hong Chen,et al.  PARA: A positive-region based attribute reduction accelerator , 2019, Inf. Sci..

[76]  Jilin Yang,et al.  Quantitative three-way class-specific attribute reducts based on region preservations , 2020, Int. J. Approx. Reason..

[77]  Concha Bielza,et al.  Decision functions for chain classifiers based on Bayesian networks for multi-label classification , 2016, Int. J. Approx. Reason..

[78]  Zhi-Hua Zhou,et al.  Facial Age Estimation by Learning from Label Distributions , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.