Input Feature Selection by Mutual Information Based on Parzen Window

Mutual information is a good indicator of relevance between variables, and have been used as a measure in several feature selection algorithms. However, calculating the mutual information is difficult, and the performance of a feature selection algorithm depends on the accuracy of the mutual information. In this paper, we propose a new method of calculating mutual information between input and class variables based on the Parzen window, and we apply this to a feature selection algorithm for classification problems.

[1]  Chong-Ho Choi,et al.  Improved mutual information feature selector for neural networks in supervised learning , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[2]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[3]  Keinosuke Fukunaga,et al.  The Reduced Parzen Classifier , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[5]  Jan Paul Siebert,et al.  Vehicle Recognition Using Rule Based Methods , 1987 .

[6]  K. Torkkola,et al.  Nonlinear feature transforms using maximum mutual information , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[7]  Octavia I. Camps,et al.  Weighted Parzen Windows for Pattern Classification , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[9]  William M. Campbell,et al.  Mutual Information in Learning Feature Transformations , 2000, ICML.

[10]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Chahab Nastar,et al.  Relevance feedback and category search in image databases , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[13]  J. Príncipe,et al.  Learning from examples with quadratic mutual information , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[14]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[17]  Yoshihiko Hamamoto,et al.  Evaluation of a modified Parzen classifier in high dimensional spaces , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[18]  Yoshihiko Hamamoto,et al.  On the Behavior of Artificial Neural Network Classifiers in High-Dimensional Spaces , 1996, IEEE Trans. Pattern Anal. Mach. Intell..