Feature selection based on survival Cauchy-Schwartz mutual information

Feature selection techniques play a crucial role in machine learning tasks such as regression and classification. Many filter methods of feature selection are based on the mutual information (e.g. MIFS, MIFS-U, NMIFS, and mRMR methods). In this work, a new mutual information is defined based on the cross survival information potential (CSIP) and Cauchy-Schwartz divergence (CSD), called the survival Cauchy-Schwartz mutual information (SCS-MI). We apply this new mutual information to select an informative subset of features for a SVM classifier. Experimental results illustrate the desirable performance of the new method.

[1]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yunmei Chen,et al.  Cumulative residual entropy: a new measure of information , 2004, IEEE Transactions on Information Theory.

[3]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Mutual Information Feature Selection , 2012 .

[4]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[5]  Baba C. Vemuri,et al.  Non-Rigid Multi-Modal Image Registration Using Cross-Cumulative Residual Entropy , 2007, International Journal of Computer Vision.

[6]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[7]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[8]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[9]  Pavel Pudil,et al.  Conditional Mutual Information Based Feature Selection for Classification Task , 2007, CIARP.

[10]  Kostas Zografos,et al.  Survival exponential entropies , 2005, IEEE Transactions on Information Theory.

[11]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[12]  Badong Chen,et al.  Survival Information Potential: A New Criterion for Adaptive System Training , 2012, IEEE Transactions on Signal Processing.