Investigating the influence of feature correlations on automatic relevance determination

Feature selection is the technique commonly used in machine learning to select a subset of relevant features for building robust learning models. Ensemble feature relevance determination can properly group the most relevant features together and separate the relevant features from the irrelevant and redundant features. However, it cannot provide reliable local feature relevance rank. In this paper, we demonstrate that the predicted local relevance rank for the relevant features could be influenced by their highly correlated redundant features, according to the strength of their correlations.

[1]  W. Wang,et al.  A Comparative Study of Feature-Salience Ranking Techniques , 2001, Neural Computation.

[2]  William D. Penny,et al.  Bayesian neural networks for classification: how useful is the evidence framework? , 1999, Neural Networks.

[3]  William B. Yates,et al.  Engineering Multiversion Neural-Net Systems , 1996, Neural Computation.

[4]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[5]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[6]  Derek Partridge.  Network generalization differences quantified , 1996, Neural Networks.

[7]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[8]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[9]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[10]  Antony Browne,et al.  Using Ensembles of Neural Networks to Improve Automatic Relevance Determination , 2007, 2007 International Joint Conference on Neural Networks.

[11]  Salim Hariri,et al.  A new dependency and correlation analysis for features , 2005, IEEE Transactions on Knowledge and Data Engineering.

[12]  B. De Moor,et al.  Engine sound comfortability: relevant sound quality parameters and classification , 2005 .

[13]  Jouko Lampinen,et al.  Bayesian approach for neural networks--review and case studies , 2001, Neural Networks.