Feature Selection via Coalitional Game Theory

We present and study the contribution-selection algorithm (CSA), a novel algorithm for feature selection. The algorithm is based on the multiperturbation shapley analysis (MSA), a framework that relies on game theory to estimate usefulness. The algorithm iteratively estimates the usefulness of features and selects them accordingly, using either forward selection or backward elimination. It can optimize various performance measures over unseen data such as accuracy, balanced error rate, and area under receiver-operator-characteristic curve. Empirical comparison with several other existing feature selection methods shows that the backward elimination variant of CSA leads to the most accurate classification results on an array of data sets.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[4]  Isaac Meilijson,et al.  Axiomatic Scalable Neurocontroller Analysis via the Shapley Value , 2006, Artificial Life.

[5]  Joan Feigenbaum,et al.  Sharing the Cost of Multicast Transmissions , 2001, J. Comput. Syst. Sci..

[6]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[7]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.

[8]  Nicholas Kushmerick,et al.  Learning to remove Internet advertisements , 1999, AGENTS '99.

[9]  Stephen Cox,et al.  Some statistical issues in the comparison of speech recognition algorithms , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[10]  Gérard Dreyfus,et al.  Ranking a Random Feature for Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[11]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[12]  A. Roth.  Axiomatic models of bargaining , 1979 .

[14]  Isaac Meilijson,et al.  Fair Attribution of Functional Contribution in Artificial and Biological Networks , 2004, Neural Computation.

[15]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[16]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[17]  Ki Hang Kim.  Game theory in the social sciences , 1986 .

[18]  O. Gefeller,et al.  Averaging attributable fractions in the multifactorial situation: assumptions and interpretation. , 1998, Journal of clinical epidemiology.

[19]  L. Shapley.  A Value for n-person Games , 1988 .

[20]  Isabelle Guyon,et al.  Design of experiments for the NIPS 2003 variable selection benchmark , 2003 .

[21]  Louis J. Billera,et al.  Internal Telephone Billing Rates - A Novel Application of Non-Atomic Game Theory , 1978, Oper. Res..

[22]  Henrik Bengtsson,et al.  The R.oo package - Object-oriented programming with references using standard R code , 2003 .

[23]  M. Sanderson.  The Reuters collection , 1994 .

[24]  James Theiler,et al.  Grafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space , 2003, J. Mach. Learn. Res..

[25]  M. Shubik.  Incentives, Decentralized Control, the Assignment of Joint Costs and Internal Pricing , 1962 .

[26]  Jun He,et al.  A unified way in incorporating segmental feature and segmental model into HMM , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[27]  Patricia S. O Sullivan,et al.  100 Statistical Tests , 1995 .

[28]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[29]  Léon Personnaz,et al.  MLPs (Mono-Layer Polynomials and Multi-Layer Perceptrons) for Nonlinear Modeling , 2003, J. Mach. Learn. Res..

[30]  Martin Shubik,et al.  A Method for Evaluating the Distribution of Power in a Committee System , 1954, American Political Science Review.