A Feature Selection Approach Based on Information Theory for Classification Tasks

This paper proposes the use of a Information Theory measure in a dynamic feature selection approach. We tested such approach including elements of Information Theory in the process, such as Mutual Information, and compared with classical methods like PCA and LDA as well as Mutual Information based algorithms. Results showed that the proposed method achieved better performance in most cases when compared with the other methods. Based on this, we could conclude that the proposed approach is very promising since it achieved better performance than well-established dimensionality reduction methods.

[1]  T. Santhanam,et al.  Application of K-Means and Genetic Algorithms for Dimension Reduction by Integrating SVM for Diabetes Diagnosis , 2015 .

[2]  I. Jolliffe.  Principal Component Analysis , 2002 .

[3]  Anil K. Jain,et al.  Simultaneous feature selection and clustering using mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[5]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[6]  Adrião Duarte Dória Neto,et al.  A Combination Method for Reducing Dimensionality in Large Datasets , 2016, ICANN.

[7]  Kewei Cheng,et al.  Feature Selection , 2016, ACM Comput. Surv..

[8]  Dahua Lin,et al.  Conditional Infomax Learning: An Integrated Framework for Feature Extraction and Fusion , 2006, ECCV.

[9]  G. McLachlan.  Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[10]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Anne M. P. Canuto,et al.  An unsupervised-based dynamic feature selection for classification tasks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[13]  Anne M. P. Canuto,et al.  Fusion Approaches of Feature Selection Algorithms for Classification Problems , 2016, 2016 5th Brazilian Conference on Intelligent Systems (BRACIS).