Combining Classifiers with Particle Swarms

Multiple classifier systems have shown a significant potential gain in comparison to the performance of an individual best classifier. In this paper, a weighted combination model of multiple classifier systems was presented, which took sum rule and majority vote as special cases. Particle swarm optimization (PSO), a new population-based evolutionary computation technique, was used to optimize the model. We referred the optimized model as PSO-WCM. An experimental investigation was performed on UCI data sets and encouraging results were obtained. PSO-WCM proposed in this paper is superior to other combination rules given larger data sets. It is also shown that rejection of weak classifier in the ensemble can improve classification performance further.

[1]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[2]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[3]  Ching Y. Suen,et al.  Multiple Classifier Combination Methodologies for Different Output Levels , 2000, Multiple Classifier Systems.

[4]  Alper Baykut,et al.  Towards Automated Classifier Combination for Pattern Recognition , 2003, Multiple Classifier Systems.

[5]  Bogdan Gabrys,et al.  Application of the Evolutionary Algorithms for Classifier Selection in Multiple Classifier Systems with Majority Voting , 2001, Multiple Classifier Systems.

[6]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[7]  A. E. Eiben,et al.  Evolutionary Programming VII , 1998, Lecture Notes in Computer Science.

[8]  Joydeep Ghosh,et al.  Multiclassifier Systems: Back to the Future , 2002, Multiple Classifier Systems.

[9]  Tin Kam Ho,et al.  Complexity of Classification Problems and Comparative Advantages of Combined Classifiers , 2000, Multiple Classifier Systems.

[10]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[11]  Peter J. Angeline,et al.  Evolutionary Optimization Versus Particle Swarm Optimization: Philosophy and Performance Differences , 1998, Evolutionary Programming.

[12]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[13]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..