Support Vector Machine Chains with a Novel Tournament Voting

Support vector machine (SVM) algorithms have been widely used for classification in many different areas. However, the use of a single SVM classifier is limited by the advantages and disadvantages of the algorithm. This paper proposes a novel method, called support vector machine chains (SVMC), which involves chaining together multiple SVM classifiers in a special structure, such that each learner is constructed by decrementing one feature at each stage. This paper also proposes a new voting mechanism, called tournament voting, in which the outputs of classifiers compete in groups, the common result in each group gradually moves to the next round, and, at the last round, the winning class label is assigned as the final prediction. Experiments were conducted on 14 real-world benchmark datasets. The experimental results showed that SVMC (88.11%) achieved higher accuracy than SVM (86.71%) on average thanks to the feature selection, sampling, and chain structure combined with multiple models. Furthermore, the proposed tournament voting demonstrated higher performance than the standard majority voting in terms of accuracy. The results also showed that the proposed SVMC method outperformed the state-of-the-art methods with a 6.88% improvement in average accuracy.

[1]  G. Gangadharan,et al.  Fair and Explainable Depression Detection in Social Media , 2023, Inf. Process. Manag..

[2]  D. Arora,et al.  An optimal method for identification of finger vein using supervised learning , 2022, Measurement: Sensors.

[3]  O. Coulon,et al.  Ensemble learning for the detection of pli-de-passages in the superior temporal sulcus , 2022, NeuroImage.

[4]  Md. Saiful Islam,et al.  A new ensemble learning approach to detect malaria from microscopic red blood cell images , 2022, Sensors International.

[5]  S. K. Himanshu,et al.  A Support Vector Machine and Image Processing based Approach for Counting Open Cotton Bolls and Estimating Lint Yield from UAV Imagery , 2022, Smart Agricultural Technology.

[6]  Yan-ching Liu.  bsnsing: A decision tree induction method based on recursive optimal boolean rule composition , 2022, INFORMS Journal on Computing.

[7]  Hatem M. Abdelkader,et al.  A novel ensemble approach for heterogeneous data with active learning , 2022, International Journal of Engineering Business Management.

[8]  Y. B. Wah,et al.  Comparison of ensemble hybrid sampling with bagging and boosting machine learning approach for imbalanced data , 2022, Indonesian Journal of Electrical Engineering and Computer Science.

[9]  Jamal Shahrabi,et al.  Bagging Supervised Autoencoder Classifier for Credit Scoring , 2021, Expert Syst. Appl..

[10]  Seokho Kang,et al.  k-Nearest Neighbor Learning with Graph Neural Networks , 2021, Mathematics.

[11]  Mr. Vijay M. Khadse Et al.,et al.  A Novel Approach of Ensemble Learning with Feature Reduction for Classification of Binary and Multiclass IoT Data , 2021 .

[12]  Artittayapron Rojarath,et al.  Cost-sensitive probability for weighted voting in an ensemble model for multi-class classification problems , 2021, Applied Intelligence.

[13]  Adeeb Noor,et al.  A Novel Approach to Ensemble Classifiers: FsBoost-Based Subspace Method , 2020, Mathematical Problems in Engineering.

[14]  Derya Birant,et al.  Enhanced Bagging (eBagging): A Novel Approach for Ensemble Learning , 2020 .

[15]  Hieu Pham,et al.  Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems , 2019, Machine Learning with Applications.

[16]  José Luis Rojo-Álvarez,et al.  Digital Signal Processing with Kernel Methods , 2018 .

[17]  Francisco Javier García Castellano,et al.  AdaptativeCC4.5: Credal C4.5 with a rough class noise estimator , 2018, Expert Syst. Appl..

[18]  Yelipe UshaRani,et al.  An efficient approach for imputation and classification of medical data values using class-based clustering of medical records , 2017, Comput. Electr. Eng..

[19]  Jane You,et al.  Progressive subspace ensemble learning , 2016, Pattern Recognit..

[20]  Francisco Herrera,et al.  An Interval Valued K-Nearest Neighbors Classifier , 2015, IFSA-EUSFLAT.

[21]  A. Sankar,et al.  Pattern Matching based Classification using Ant Colony Optimization based Feature Selection , 2015, Appl. Soft Comput..

[22]  Halil Yigit,et al.  ABC-based distance-weighted kNN algorithm , 2015, J. Exp. Theor. Artif. Intell..

[23]  Wuyi Yue,et al.  Support vector machine based multiagent ensemble learning for credit risk evaluation , 2010, Expert Syst. Appl..

[24]  Israa M. Hayder,et al.  Predicting reaction based on customer's transaction using machine learning approaches , 2023, International Journal of Electrical and Computer Engineering (IJECE).

[25]  M. Vázquez,et al.  A quick method for fraud detection in egg labels based on egg centrifugation plasma. , 2023, Food chemistry.

[26]  Shaha T. Al-Otaibi,et al.  An Intelligent Hazardous Waste Detection and Classification Model Using Ensemble Learning Techniques , 2023, Computers, Materials & Continua.

[27]  Muawia A. Elsadig,et al.  Breast cancer detection using machine learning approaches: a comparative study , 2023, International Journal of Electrical and Computer Engineering (IJECE).

[28]  Ibomoiye Domor Mienye,et al.  A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects , 2022, IEEE Access.

[29]  Changjian Zhou,et al.  COVID-19 Detection Based on Image Regrouping and Resnet-SVM Using Chest X-Ray Images , 2021, IEEE Access.

[30]  K. Yen,et al.  Rank-Based Chain-Mode Ensemble for Binary Classification , 2020 .

[31]  Sebastián Ventura,et al.  A gene expression programming algorithm for discovering classification rules in the multi-objective space , 2018, Int. J. Comput. Intell. Syst..

[32]  José Luis Rojo-Álvarez,et al.  Support Vector Machine and Kernel Classification Algorithms , 2018 .

[33]  Francisco Herrera,et al.  Evolutionary wrapper approaches for training set selection as preprocessing mechanism for support vector machines: Experimental evaluation and support vector analysis , 2016, Appl. Soft Comput..

[34]  Hakan Altiotançay.  Improving the k-nearest neighbour rule: using geometrical neighbourhoods and manifold-based metrics , 2011, Expert Syst. J. Knowl. Eng..

[35]  Y. Benmahamed,et al.  Fault Diagnosis in Robot Manipulators Using SVM and KNN , 2022, Intelligent Automation & Soft Computing.