Multi-view ensemble learning using multi-objective particle swarm optimization for high dimensional data classification

Abstract In state-of-the-art, it has proven that multi-view ensemble learning performs better than classical machine learning algorithms, with the optimized setting of views (su the dataset. Therefore, it is highly required to consider a smaller number of views with higher accuracy for optimal performance of MEL. In this work, MEL using Multi-Objective Particle Swarm Optimization (MEL-MOPSO) method has been proposed. The two objectives (number of views of the data and classification accuracy of MEL) have considered where the trade-off between objectives has been performed while searching for an optimal solution using Particle Swarm Optimization (PSO) in the process of multiobjective optimization. The experiments have been done over sixteen high-dimensional datasets using four state-of-art view construction methods. The individual views of the dataset has been utilized to learn through a support vector machine algorithm. The quantitative and non-parametric statistical analyses show that the proposed method has performed effectively and efficiently.

[1]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[2]  Qingxiang Wu,et al.  Multiknowledge for decision making , 2005, Knowledge and Information Systems.

[3]  Yao Liu,et al.  Using contextual features and multi-view ensemble learning in product defect identification from online discussion forums , 2018, Decis. Support Syst..

[4]  Lior Rokach,et al.  Genetic algorithm-based feature set partitioning for classification problems , 2008, Pattern Recognit..

[5]  Trevor Darrell,et al.  Co-training with noisy perceptual observations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Qinghai Bai,et al.  Analysis of Particle Swarm Optimization Algorithm , 2010, Comput. Inf. Sci..

[7]  Xin Ye,et al.  Multi-view ensemble learning method for microblog sentiment classification , 2021, Expert Syst. Appl..

[8]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[9]  Qinghua Hu,et al.  EROS: Ensemble rough subspaces , 2007, Pattern Recognit..

[10]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[11]  Pong C. Yuen,et al.  A Boosted Co-Training Algorithm for Human Action Recognition , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Mykola Pechenizkiy,et al.  Diversity in search strategies for ensemble feature selection , 2005, Inf. Fusion.

[13]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[14]  Vipin Kumar,et al.  A Multi-view Convolutional Neural Network Approach for Image Data Classification , 2021, 2021 International Conference on Communication information and Computing Technology (ICCICT).

[15]  Shiliang Sun,et al.  View Construction for Multi-view Semi-supervised Learning , 2011, ISNN.

[16]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Horst Bunke,et al.  Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition , 2004 .

[19]  Songcan Chen,et al.  A novel multi-view learning developed from single-view patterns , 2011, Pattern Recognit..

[20]  Junfeng Wang,et al.  Improving malware detection using multi-view ensemble learning , 2016, Secur. Commun. Networks.

[21]  Sanjoy Dasgupta,et al.  PAC Generalization Bounds for Co-training , 2001, NIPS.

[22]  Yixin Chen,et al.  Automatic Feature Decomposition for Single View Co-training , 2011, ICML.

[23]  Craig A. Knoblock,et al.  Adaptive View Validation: A First Step Towards Automatic View Detection , 2002, ICML.

[24]  Melba M. Crawford,et al.  Multi-view adaptive disagreement based active learning for hyperspectral image classification , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[25]  Daren Yu,et al.  Constructing Rough Decision Forests , 2005, RSFDGrC.

[26]  Vivek Kumar Singh,et al.  A novel approach for classification of mental tasks using multiview ensemble learning (MEL) , 2020, Neurocomputing.

[27]  Francis K. H. Quek,et al.  Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets , 2003, Pattern Recognit..

[28]  Kalyanmoy Deb,et al.  Multi-objective Optimisation Using Evolutionary Algorithms: An Introduction , 2011, Multi-objective Evolutionary Optimisation for Product Design and Manufacturing.

[29]  R. Bharat Rao,et al.  Bayesian Co-Training , 2007, J. Mach. Learn. Res..

[30]  J. Pitman.  Some Probabilistic Aspects of Set Partitions , 1997 .

[31]  Carlos A. Coello Coello,et al.  A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques , 1999, Knowledge and Information Systems.