A Stable Instance Based Filter for Feature Selection in Small Sample Size Data Sets

In supervised feature selection applications it is common to have high dimensional data, but it is sometimes not easy to collect a large number of examples to represent each pattern or object class. Hence, learning in the small sample case is of practical interest. One reason for this is the difculty in collecting data for each object. We propose a filter approach for feature selection based on instance learning. Its main challenge is that it convert the problem of the small sample size to a tool that allows choosing only a few subsets of features to be combined in order to select the most relevant ones. Each instance proposes a candidate subset of the most relevant features for this instance. Small sample size makes this process feasible. Thus the high dimensionality of data is reduced to few subsets of features which number corresponds to the data sample size and this is when small sample size is of benefit to feature selection process. The combination scheme used for this purpose aims at obtaining a feature selection that yields to good classification performance while beeing stable.