Data Compression Measures for Meta-Learning Systems

An important issue in predictive modeling is model selection. This process is time consuming and can be simplified with meta-learning. However, meta-learning systems need appropriate data descriptors for proper functioning. One of them are data compression measures which can be extracted out of the instance selection methods. When we only need to estimate the classification accuracy of the model, the compression obtained from instance selection is a good approximator, but when we need to estimate other performance measures such as the precision and sensitivity then the quality of the estimated performance drops. To overcome this issue we propose a new type of compression measure: the balanced compression which is sensitive to the class label distribution and shows high correlation with precision and sensitivity of the final classifiers. We also show that the application of the balanced compression as a meta-learning descriptor allows for precise assessment of the model performance, as proved by the presented experimental evaluation.