An Effective Feature Selection Method Using Dynamic Information Criterion

With rapid development of information technology, dimensionality of data in many applications is getting higher and higher. However, many features in the high-dimensional data are redundant. Their presence may pose a great number of challenges to traditional learning algorithms. Thus, it is necessary to develop an effective technique to remove irrelevant features from data. Currently, many endeavors have been attempted in this field. In this paper, we propose a new feature selection method by using conditional mutual information estimated dynamically. Its advantage is that it can exactly represent the correlation between features along with the selection procedure. Our performance evaluations on eight benchmark datasets show that our proposed method achieves comparable performance to other well-established feature selection algorithms in most cases.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[3]  José Manuel Benítez,et al.  Consistency measures for feature selection , 2008, Journal of Intelligent Information Systems.

[4]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[5]  Lei Liu,et al.  Feature Selection Using Mutual Information: An Experimental Study , 2008, PRICAI.

[6]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[7]  Pavel Pudil,et al.  Conditional Mutual Information Based Feature Selection for Classification Task , 2007, CIARP.

[8]  Zhi-Hua Zhou,et al.  PRICAI 2008: Trends in Artificial Intelligence, 10th Pacific Rim International Conference on Artificial Intelligence, Hanoi, Vietnam, December 15-19, 2008. Proceedings , 2008, PRICAI.

[9]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[10]  Lei Liu,et al.  Ensemble gene selection by grouping for microarray data classification , 2010, J. Biomed. Informatics.

[11]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Chong-Ho Choi,et al.  Input Feature Selection by Mutual Information Based on Parzen Window , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Lei Liu,et al.  Feature selection with dynamic mutual information , 2009, Pattern Recognit..

[14]  David A. Bell,et al.  A Formalism for Relevance and Its Application in Feature Subset Selection , 2000, Machine Learning.

[15]  José Francisco Martínez-Trinidad,et al.  Progress in Pattern Recognition, Image Analysis and Applications, 12th Iberoamericann Congress on Pattern Recognition, CIARP 2007, Valparaiso, Chile, November 13-16, 2007, Proceedings , 2008, CIARP.

[16]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[17]  Shimon Ullman,et al.  Learning to classify by ongoing feature selection , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[18]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[19]  Melanie Hilario,et al.  Approaches to dimensionality reduction in proteomic biomarker studies , 2007, Briefings Bioinform..

[20]  David G. Stork,et al.  Pattern Classification , 1973 .

[21]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..