KNN-based multi-label twin support vector machine with priority of labels

Abstract A common approach to solve multi-label classification problems is the transformation method, in which a multi-label problem is converted into multiple single-label representations. With an efficient implementation of single-label algorithms, and considering dependency between labels and the fact that similar samples often share the same labels, we can expect a highly effective classification in multi-label datasets. In this paper, to tackle multi-label classification problem, first using an improved twin support vector machine classifier, the hyperplanes containing structural information of samples and local information of each class label, are found. Then the prior probability of each hyperplane and the sample points that are located in the margins of the hyperplanes are extracted. For the prediction phase, several facts are applied to help finding the sets of relevant labels of a sample: (1) samples with similar labels share the same information, (2) local information has a great impact on the performance and efficiency of a multi-label algorithm, and (3) the samples that are most important in classifications are located in the margin of hyperplanes. To obtain the sets of relevant labels for a test sample, first its k nearest samples in the margin space of the hyperplanes are found. Then, relevant labels are extracted using statistical and membership counting methods. The nonlinear version of the algorithm is also developed through kernel trick. The experimental results obtained from different datasets and different measures indicated good performances of the proposed algorithm, compared to several relevant methods.

[1]  Daniel S. Yeung,et al.  Structured large margin machines: sensitive to data distributions , 2007, Machine Learning.

[2]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[3]  Jianhua Xu,et al.  An efficient multi-label support vector machine with a zero label , 2012, Expert Syst. Appl..

[4]  Xianli Pan,et al.  Safe Screening Rules for Accelerating Twin Support Vector Machine Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Ying Zhang,et al.  Semi-supervised support vector classification with self-constructed Universum , 2016, Neurocomputing.

[6]  Lan Bai,et al.  Stochastic Gradient Twin Support Vector Machine for Large Scale Problems , 2017, Inf. Sci..

[7]  David Zhang,et al.  Multi-Label Dictionary Learning for Image Annotation , 2016, IEEE Transactions on Image Processing.

[8]  Michael K. Ng,et al.  ML-FOREST: A Multi-Label Tree Ensemble Method for Multi-Label Classification , 2016, IEEE Transactions on Knowledge and Data Engineering.

[9]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[10]  Reshma Khemchandani,et al.  Fuzzy least squares twin support vector clustering , 2016, Neural Computing and Applications.

[11]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[12]  Muhammad Tanveer,et al.  A robust fuzzy least squares twin support vector machine for class imbalance learning , 2018, Appl. Soft Comput..

[13]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[15]  Lei Tang,et al.  Large scale multi-label classification via metalabeler , 2009, WWW '09.

[16]  Min-Ling Zhang,et al.  Ml-rbf: RBF Neural Networks for Multi-Label Learning , 2009, Neural Processing Letters.

[17]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[18]  Xianli Pan,et al.  K-nearest neighbor based structural twin support vector machine , 2015, Knowl. Based Syst..

[19]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[20]  Concha Bielza,et al.  Multi-label classification with Bayesian network-based chain classifiers , 2014, Pattern Recognit. Lett..

[21]  Shangbing Gao,et al.  Weighted Twin Support Vector Machines with Local Information and its application , 2012, Neural Networks.

[22]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[23]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[24]  Xinjun Peng,et al.  PTSVRs: Regression models via projection twin support vector machine , 2018, Inf. Sci..

[25]  Yuan-Hai Shao,et al.  An efficient weighted Lagrangian twin support vector machine for imbalanced data classification , 2014, Pattern Recognit..

[26]  Witold Pedrycz,et al.  Neighborhood rough sets based multi-label classification for automatic image annotation , 2013, Int. J. Approx. Reason..

[27]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Rommel M. Barbosa,et al.  Improvements on least squares twin multi-class classification support vector machine , 2018, Neurocomputing.

[29]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[30]  Luca Martino,et al.  Multi-label methods for prediction with sequential data , 2015, Pattern Recognit..

[31]  Eyke Hüllermeier,et al.  A Unified Model for Multilabel Classification and Ranking , 2006, ECAI.

[32]  Saso Dzeroski,et al.  Predicting gene function using hierarchical multi-label decision tree ensembles , 2010, BMC Bioinformatics.

[33]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[34]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[35]  Eyke Hüllermeier,et al.  Combining Instance-Based Learning and Logistic Regression for Multilabel Classification , 2009, ECML/PKDD.

[36]  Saso Dzeroski,et al.  Decision trees for hierarchical multi-label classification , 2008, Machine Learning.

[37]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[38]  Qiang Yang,et al.  Discriminatively regularized least-squares classification , 2009, Pattern Recognit..

[39]  Lan Bai,et al.  Twin Support Vector Machine for Clustering , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[40]  M. Monica Subashini,et al.  A non-invasive methodology for the grade identification of astrocytoma using image processing and artificial intelligence techniques , 2016, Expert Syst. Appl..

[41]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[42]  Josef Kittler,et al.  Multi-label classification using stacked spectral kernel discriminant analysis , 2016, Neurocomputing.

[43]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[44]  Yuan-Hai Shao,et al.  MLTSVM: A novel twin support vector machine to multi-label learning , 2016, Pattern Recognit..