Iterative Relief

Feature weighting algorithms assign weights to features according to their relevance to a particular task. Unfortunately, the best-known feature weighting algorithm, ReliefF, is biased. It decreases the relevance of some features and increases the relevance of others when irrelevant attributes are added to the data set. This paper presents an improved version of the algorithm, Iterative Relief, and shows on synthetic data that it removes the bias found in ReliefF. This paper also shows that Iterative Relief outperforms ReliefF on the task of cat and dog discrimination, using real images.

[1]  Hiroshi Ishiguro,et al.  Robot oriented state space construction , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[2]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[3]  Bruce Draper,et al.  Feature selection from huge feature sets in the context of computer vision , 2000 .

[4]  Marko Robnik-Sikonja,et al.  Comprehensible Interpretation of Relief's Estimates , 2001, ICML.

[5]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[6]  Bruce A. Draper,et al.  Feature selection from huge feature sets , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.