New Representations in PSO for Feature Construction in Classification

Feature construction can improve the classification performance by constructing high-level features using the original low-level features and function operators. Particle swarm optimisation (PSO) is an powerful global search technique, but it cannot be directly used for feature construction because of its representation scheme. This paper proposes two new representations, pair representation and array representation, which allow PSO to direct evolve function operators. Two PSO based feature construction algorithms (PSOFCPair and PSOFCArray) are then developed. The two new algorithms are examined and compared with the first PSO based feature construction algorithm (PSOFC), which employs an inner loop to select function operators. Experimental results show that both PSOFCPair and PSOFCArray can increase the classification performance by constructing a new high-level feature. PSOFCArray outperforms PSOFCPair and achieves similar results to PSOFC, but uses significantly shorter computational time. This paper represents the first work on using PSO to directly evolve function operators for feature construction.

[1]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[2]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[3]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[4]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[5]  George D. Smith,et al.  Evolutionary Feature Construction Using Information Gain and Gini Index , 2004, EuroGP.

[6]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[7]  Kourosh Neshatian,et al.  Feature Manipulation with Genetic Programming , 2010 .

[8]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[9]  Mengjie Zhang,et al.  PSO for feature construction and binary classification , 2013, GECCO '13.

[10]  Mark Johnston,et al.  Feature Construction and Dimension Reduction Using Genetic Programming , 2007, Australian Conference on Artificial Intelligence.

[11]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[12]  George D. C. Cavalcanti,et al.  An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting , 2007, 2007 IEEE Congress on Evolutionary Computation.

[13]  Georgios Dounias,et al.  Particle swarm optimization for pap-smear diagnosis , 2008, Expert Syst. Appl..

[14]  Alan F. Murray,et al.  IEEE International Conference on Neural Networks , 1997 .

[15]  Mengjie Zhang,et al.  A multi-objective particle swarm optimisation for filter-based feature selection in classification problems , 2012, Connect. Sci..

[16]  Mengjie Zhang,et al.  A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming , 2012, IEEE Transactions on Evolutionary Computation.

[17]  Krzysztof Krawiec,et al.  Genetic Programming-based Construction of Features for Machine Learning and Knowledge Discovery Tasks , 2002, Genetic Programming and Evolvable Machines.