PSO for feature construction and binary classification

In classification, the quality of the data representation significantly influences the performance of a classification algorithm. Feature construction can improve the data representation by constructing new high-level features. Particle swarm optimisation (PSO) is a powerful search technique, but has never been applied to feature construction. This paper proposes a PSO based feature construction approach (PSOFC) to constructing a single new high-level feature using original low-level features and directly addressing binary classification problems without using any classification algorithm. Experiments have been conducted on seven datasets of varying difficulty. Three classification algorithms (decision trees, naive bayes, and k-nearest neighbours) are used to evaluate the performance of the constructed feature on test set. Experimental results show that a classification algorithm using the single constructed feature often achieves similar (or even better) classification performance than using all the original features, and in almost all cases, adding the constructed feature to the original features significantly improves its classification performance. In most cases, PSOFC as a classification algorithm (using the constructed feature only) achieves better classification performance than a classification algorithm using all the original features, but needs much less computational cost. This paper represents the first study on using PSO for feature construction in classification.