Searching optimal feature subset using mutual information

A novel feature selection methodology is proposed with the concept of mutual information. The proposed methodology effectively circumvents two major problems in feature selection process: to identify the irrelevancy and redundancy in the feature set, and to estimate the optimal feature subset for classification task.

[1]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[2]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[3]  Wentian Li.  Mutual information functions versus correlation functions , 1990 .

[4]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[5]  A. S. Weigend,et al.  Selecting Input Variables Using Mutual Information and Nonparemetric Density Estimation , 1994 .

[6]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[7]  Moon,et al.  Estimation of mutual information using kernel density estimators. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[8]  Mohamed A. Deriche,et al.  An optimal feature selection technique using the concept of mutual information , 2001, Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467).

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Huan Liu,et al.  A Monotonic Measure for Optimal Feature Selection , 1998, ECML.

[11]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[12]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..