Binary particle swarm optimisation and rough set theory for dimension reduction in classification

Dimension reduction plays an important role in many classification tasks. In this work, we propose a new filter dimension reduction algorithm (PSOPRSE) using binary particle swarm optimisation and probabilistic rough set theory. PSOPRSE aims to maximise a classification performance measure and minimise a newly developed measure reflecting the number of attributes. Both measures are formed by probabilistic rough set theory. PSOPRSE is compared with two existing PSO based algorithms and two traditional filter dimension reduction algorithms on six discrete datasets of varying difficulty. Five continues datasets including a large number of attributes are discretised and used to further examine the performance of PSOPRSE. Three learning algorithms, namely decision trees, nearest neighbour algorithms and naive Bayes, are used in the experiments to examine the generality of PSOPRSE. The results show that PSOPRSE can significantly decrease the number of attributes and maintain or improve the classification performance over using all attributes. In most cases, PSOPRSE outperforms the first PSO based algorithm and achieves better or much better classification performance than the second PSO based algorithm and the two traditional methods, although the number of attributes is slightly large in some cases. The results also show that PSOPRSE is general to the three different classification algorithms.

[1]  Hao Dong,et al.  An improved particle swarm optimization for feature selection , 2011 .

[2]  Georgios Dounias,et al.  Particle swarm optimization for pap-smear diagnosis , 2008, Expert Syst. Appl..

[3]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[4]  Wei-Chang Yeh,et al.  Feature selection with Intelligent Dynamic Swarm and Rough Set , 2010, Expert Syst. Appl..

[5]  Yahya Slimani,et al.  Adaptive Particle Swarm Optimizer for Feature Selection , 2010, IDEAL.

[6]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[7]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[8]  Mengjie Zhang,et al.  Genetic Programming for Feature Ranking in Classification Problems , 2008, SEAL.

[9]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[10]  Siti Mariyam Shamsuddin,et al.  Quantum-inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks , 2011 .

[11]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[12]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[13]  Shih-Wei Lin,et al.  PSOLDA: A particle swarm optimization approach for enhancing classification accuracy rate of linear discriminant analysis , 2009, Appl. Soft Comput..

[14]  Mengjie Zhang,et al.  Genetic Programming for Feature Subset Ranking in Binary Classification Problems , 2009, EuroGP.

[15]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[16]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[17]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[18]  Mengjie Zhang,et al.  A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming , 2012, IEEE Transactions on Evolutionary Computation.

[19]  Zexuan Zhu,et al.  Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[21]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[22]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[23]  Ian H. Witten,et al.  Data mining - practical machine learning tools and techniques, Second Edition , 2005, The Morgan Kaufmann series in data management systems.

[24]  He Ming.  A Rough Set Based Hybrid Method to Feature Selection , 2008, 2008 International Symposium on Knowledge Acquisition and Modeling.

[25]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[26]  Yiyu Yao,et al.  Attribute reduction in decision-theoretic rough set models , 2008, Inf. Sci..

[27]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[28]  Mengjie Zhang,et al.  A Dimension Reduction Approach to Classification Based on Particle Swarm Optimisation and Rough Set Theory , 2012, Australasian Conference on Artificial Intelligence.

[29]  Mark Johnston,et al.  Particle Swarm Optimization based Adaboost for face detection , 2009, 2009 IEEE Congress on Evolutionary Computation.

[30]  Andreas König,et al.  Feature-Level Fusion by Multi-Objective Binary Particle Swarm Based Unbiased Feature Selection for Optimized Sensor System Design , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.