Applying Mutual Information for Prototype or Instance Selection in Regression Problems

1- University of Ja´en - Department of InformaticsSpain2- University of Granada - Department of Computer Technology and ArchitectureSpain3- Helsinki University of Technology - Information and Computer Science DepartmentFinlandAbstract. The problem of selecting the patterns to be learned by anymodel is usually not considered by the time of designing the concretemodel but as a preprocessing step. Information theory provides a robusttheoretical framework for performing input variable selection thanks to theconcept of mutual information. Recently the computation of the mutualinformation for regression tasks has been proposed so this paper presentsa new application of the concept of mutual information not to select thevariables but to decide which prototypes should belong to the training dataset in regression problems. The proposed methodology consists in decidingif a prototype should belong or not to the training set using as criteria theestimation of the mutual information between the variables. The noveltyof the approach is to focus in prototype selection for regression problemsinstead of classification as the majority of the literature deals only withthe last one. Other element that distinguishes this work from others isthat it is not proposed as an outlier identificator but as an algorithm thatdetermines the best subset of input vectors by the time of building a modelto approximate it. As the experiment section shows, this new method isable to identify a high percentage of the real data set when it is appliedto a highly distorted data sets.