The filter is a simple and practical method for feature selection, but it can introduce biases resulting in decreased prediction performance. We propose an enhanced filter method that exploits features from two information-based filtering steps: supervised and unsupervised. By combining the features in these steps we attempt to reduce biases caused by misleading causal relations induced in the supervised selection procedure. When tested with the five datasets given at the NIPS 2003 Feature Extraction Workshop, our approach attained a significant performance, considering the simplicity of the approach. We expect the combined information-based method to be a promising substitute for classical filter methods.
[1]
Trevor Hastie,et al.
The Elements of Statistical Learning
,
2001
.
[2]
David G. Stork,et al.
Pattern Classification
,
1973
.
[3]
Isabelle Guyon,et al.
An Introduction to Variable and Feature Selection
,
2003,
J. Mach. Learn. Res..
[4]
Bernhard E. Boser,et al.
A training algorithm for optimal margin classifiers
,
1992,
COLT '92.
[5]
Daphne Koller,et al.
Toward Optimal Feature Selection
,
1996,
ICML.
[6]
P. Spirtes,et al.
Causation, prediction, and search
,
1993
.