Twin support vector machines with privileged information

Abstract In the field of machine learning, collected data always have additional features which are always referred as privileged information. Privileged information learning is mainly used to help train the classifier in the training process, and predict the unseen example by the learned classifier. In this paper, we propose a new method named twin support vector machines with privileged information (TWSVM-PI). In the proposed method, we first introduce the privileged information into twin SVMs so as to construct a model for prediction, and then utilize the Lagrangian multiplier method to optimize the proposed objective function. Thus, we obtain two nonparallel classification hyperplanes by solving two smaller sized quadratic programming problems (QPPs), which can shorter the computational time and improve the accuracy of the prediction. Finally, we conduct extensive experiments to evaluate the performance of the proposed TWSVM-PI method. The results have shown that our proposed method can obtain a better performance compared with state-of-the-art methods.

[1]  Hong Shen,et al.  Imbalanced data classification based on hybrid resampling and twin support vector machine , 2017, Comput. Sci. Inf. Syst..

[2]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[3]  Marcos Aurélio Domingues,et al.  Privileged Information for Hierarchical Document Clustering: A Metric Learning Approach , 2014, 2014 22nd International Conference on Pattern Recognition.

[4]  Ionut Emil Iacob,et al.  DCSVM: fast multi-class classification using support vector machines , 2018, International Journal of Machine Learning and Cybernetics.

[5]  Deepak Gupta,et al.  Entropy based fuzzy least squares twin support vector machine for class imbalance learning , 2018, Applied Intelligence.

[6]  Nasser Ghadiri,et al.  Fuzzy Least Squares Twin Support Vector Machines , 2015, Eng. Appl. Artif. Intell..

[7]  Xi-Zhao Wang,et al.  Intuitionistic Fuzzy Twin Support Vector Machines , 2019, IEEE Transactions on Fuzzy Systems.

[8]  Wu Tie-jun.  Support vector machines for pattern recognition , 2003 .

[9]  Kup-Sze Choi,et al.  Least squares support vector machines with fast leave-one-out AUC optimization on imbalanced prostate cancer data , 2020, Int. J. Mach. Learn. Cybern..

[10]  Jia Yu,et al.  KNN-based weighted rough ν-twin support vector machine , 2014, Knowl. Based Syst..

[11]  Lan Bai,et al.  Twin Support Vector Machine for Clustering , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Yu Guo,et al.  Learning using privileged information for HRRP-based radar target recognition , 2018, IET Signal Process..

[13]  Lidong Wang,et al.  Wavelet transform-based weighted $$\nu$$ν-twin support vector regression , 2019, Int. J. Mach. Learn. Cybern..

[14]  Muhammad Tanveer,et al.  A robust fuzzy least squares twin support vector machine for class imbalance learning , 2018, Appl. Soft Comput..

[15]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[16]  Wenjian Wang,et al.  Granular support vector machine: a review , 2017, Artificial Intelligence Review.

[17]  Ting Zhu,et al.  A new twin support vector machine for pattern recognition , 2016, 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[18]  Changyin Sun,et al.  Discriminative Multi-View Privileged Information Learning for Image Re-Ranking , 2018, IEEE Transactions on Image Processing.

[19]  Philip S. Yu,et al.  A Framework for Clustering Uncertain Data Streams , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[20]  Mohammad Saraee,et al.  Simulated annealing least squares twin support vector machine (SA-LSTSVM) for pattern classification , 2017, Soft Comput..

[21]  Ning Ye,et al.  Robust capped L1-norm twin support vector machine , 2019, Neural Networks.

[22]  Bernt Schiele,et al.  Learning using privileged information: SV M+ and weighted SVM , 2013, Neural Networks.

[23]  Qiang Ji,et al.  Learning with privileged information using Bayesian networks , 2015, Frontiers of Computer Science.

[24]  Lan Bai,et al.  Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding , 2019, Knowl. Based Syst..

[25]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Suresh Chandra,et al.  Robust Parametric Twin Support Vector Machine for Pattern Classification , 2018, Neural Processing Letters.

[27]  Aruna Tiwari,et al.  KOC+: Kernel ridge regression based one-class classification using privileged information , 2019, Inf. Sci..

[28]  Jalal A. Nasiri,et al.  KNN-based least squares twin support vector machine for pattern classification , 2018, Applied Intelligence.

[29]  Rauf Izmailov,et al.  Learning with Privileged Information for Improved Target Classification , 2014, Int. J. Monit. Surveillance Technol. Res..

[30]  Fan Meng,et al.  Pedestrian detection based on the privileged information , 2018, Neural Computing and Applications.

[31]  Xizhao Wang,et al.  Erratum to "Entropy-based fuzzy support vector machine for imbalanced datasets" [Knowl.-Based Syst. 115 (2017) 87-99] , 2020, Knowl. Based Syst..

[32]  Yuan-Hai Shao,et al.  Robust Rescaled Hinge Loss Twin Support Vector Machine for Imbalanced Noisy Classification , 2019, IEEE Access.

[33]  Evgeny Burnaev,et al.  Anomaly Pattern Recognition with Privileged Information for Sensor Fault Detection , 2018, ANNPR.

[34]  Hossein Karshenas,et al.  KNN-based multi-label twin support vector machine with priority of labels , 2018, Neurocomputing.

[35]  Dong Xu,et al.  Twin support vector hypersphere (TSVH) classifier for pattern recognition , 2013, Neural Computing and Applications.

[36]  Shiyu Chen,et al.  Learning with Privileged Information for Multi-Label Classification , 2017, Pattern Recognit..

[37]  Yi Yang,et al.  Image Classification by Cross-Media Active Learning With Privileged Information , 2016, IEEE Transactions on Multimedia.

[38]  Ricardo M. Marcacini,et al.  Incremental hierarchical text clustering with privileged information , 2013, ACM Symposium on Document Engineering.

[39]  Meng Wang,et al.  Person Re-Identification With Metric Learning Using Privileged Information , 2018, IEEE Transactions on Image Processing.

[40]  Nan Zhang,et al.  Twin support vector machine: theory, algorithm and applications , 2017, Neural Computing and Applications.

[41]  Jun Zhang,et al.  Sparse and heuristic support vector machine for binary classifier and regressor fusion , 2019, International Journal of Machine Learning and Cybernetics.