Support vector machines with the known feature-evolution priors

Abstract In traditional SVMs (support vector machines), each feature involved in an object is assumed to contribute equally to the classification accuracy. However, the equal importance assumption for each feature is inappropriate for classifying the objects. The existing weighting strategies usually utilize the fixed scaling factors to measure the importance of each feature and hence they are not good fits for feature evolution. In this paper, with the assumption that a K nown F eature- E volution P rior (KFEP) can be employed to capture importance distribution from the perspective of both feature granularity and object granularity, SVM-based machines called KFEP-SVMs are developed to capture the importance of features dynamically. Also, it is revealed that SVMs with the known feature-evolution priors are equivalent to SVMs on a higher feature space. Besides, we theoretically demonstrate that the proposed machines KFEP-SVMs with the known feature-evolution prior indeed have enhanced generalization ability, compared with the classic SVMs. Experimental results show the effectiveness of the proposed machines KFEP-SVMs.

[1]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[2]  R. L. Mills,et al.  Melting properties of He3 and He4 up to 3500 kg/cm2 , 1959 .

[3]  K. Watanabe,et al.  Topology Optimization Based on Immune Algorithm and Multi-Grid Method , 2006, 2006 12th Biennial IEEE Conference on Electromagnetic Field Computation.

[4]  Chongzhao Han,et al.  Gaussian mixture implementation of PHD filter based on Dirichlet distribution , 2013 .

[5]  Julio López,et al.  Dealing with high-dimensional class-imbalanced datasets: Embedded feature selection for SVM classification , 2018, Appl. Soft Comput..

[6]  Youxi Wu,et al.  Classification of Mental Task From EEG Signals Using Immune Feature Weighted Support Vector Machines , 2011, IEEE Transactions on Magnetics.

[7]  Grigorios Tsoumakas,et al.  Tracking recurring contexts using ensemble classifiers: an application to email filtering , 2009, Knowledge and Information Systems.

[8]  Kup-Sze Choi,et al.  Output based transfer learning with least squares support vector machine and its application in bladder cancer prognosis , 2020, Neurocomputing.

[9]  Markus Flierl,et al.  Bayesian estimation of Dirichlet mixture model with variational inference , 2014, Pattern Recognit..

[10]  Aggelos K. Katsaggelos,et al.  Visualization of feature evolution during convolutional neural network training , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[11]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[12]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[13]  Paul F. Whelan,et al.  Integration of feature distributions for colour texture segmentation , 2004, ICPR 2004.

[14]  Chia-Feng Juang,et al.  Fuzzy System Learned Through Fuzzy Clustering and Support Vector Machine for Human Skin Color Segmentation , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[15]  Feiping Nie,et al.  Multi-Class L2,1-Norm Support Vector Machine , 2011, 2011 IEEE 11th International Conference on Data Mining.

[16]  Qian Zhang,et al.  A 3D model feature extraction method using curvature-based shape distribution , 2015, 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[17]  T. Minka.  Estimating a Dirichlet distribution , 2012 .

[18]  Kan Li,et al.  A deeper graph neural network for recommender systems , 2019, Knowl. Based Syst..

[19]  Grigorios Tsoumakas,et al.  On the Utility of Incremental Feature Selection for the Classification of Textual Data Streams , 2005, Panhellenic Conference on Informatics.

[20]  Hisao Ishibuchi,et al.  Deep Takagi–Sugeno–Kang Fuzzy Classifier With Shared Linguistic Fuzzy Rules , 2018, IEEE Transactions on Fuzzy Systems.

[21]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[22]  Bin Jiang,et al.  Constrained Multiobjective Optimization Algorithm Based on Immune System Model , 2016, IEEE Transactions on Cybernetics.

[23]  R. Farouki,et al.  Flux considerations in the coupling of Monte Carlo plasma sheath simulations with feature evolution models , 1992 .

[24]  Li Wang,et al.  Hybrid huberized support vector machines for microarray classification , 2007, ICML '07.

[25]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[26]  Kup-Sze Choi,et al.  A Transfer-Based Additive LS-SVM Classifier for Handling Missing Data , 2020, IEEE Transactions on Cybernetics.

[27]  Shahnorbanun Sahran,et al.  Binary biogeography-based optimization based SVM-RFE for feature selection , 2021, Appl. Soft Comput..

[28]  C. Abrams,et al.  Feature evolution simulations of copper seed layer deposition using atomic-level particle scattering information , 1999 .

[29]  Ivor W. Tsang,et al.  Generalized Multiple Kernel Learning With Data-Dependent Priors , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Demetris Stathakis,et al.  Feature Evolution for Classification of Remotely Sensed Data , 2007, IEEE Geoscience and Remote Sensing Letters.

[31]  Kup-Sze Choi,et al.  Deep Additive Least Squares Support Vector Machines for Classification With Model Transfer , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[32]  Guizhi Xu,et al.  Tumor Detection in MR Images Using One-Class Immune Feature Weighted SVMs , 2011, IEEE Transactions on Magnetics.

[33]  Amir Abbas Najafi,et al.  Polar support vector machine: Single and multiple outputs , 2016, Neurocomputing.

[34]  Roger Tan,et al.  Improved micro-Doppler features extraction using Smoothed-Pseudo Wigner-Ville Distribution , 2016, 2016 IEEE Region 10 Conference (TENCON).

[35]  R. Cluckers,et al.  Loci of Integrability, Zero Loci, and Stability Under Integration for Constructible Functions on Euclidean Space with Lebesgue Measure , 2011 .

[36]  Suresh Chandra,et al.  Large-Scale Twin Parametric Support Vector Machine Using Pinball Loss Function , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[38]  Nizar Bouguila,et al.  A Dirichlet Process Mixture of Generalized Dirichlet Distributions for Proportional Data Modeling , 2010, IEEE Transactions on Neural Networks.

[39]  Xiang Xie,et al.  Design of a Flexible Wearable Smart sEMG Recorder Integrated Gradient Boosting Decision Tree Based Hand Gesture Recognition , 2019, IEEE Transactions on Biomedical Circuits and Systems.