Efficient Feature Construction by Meta Learning – Guiding the Search in Meta Hypothesis Space

Choosing the right internal representation of examples and hypothesis is a key issue for many learning problems. Feature construction is an approach to find such a representation independently of the underlying learning algorithm. Unfortunately, the construction of features usually implies searching a very large space of possibilities and is often computationally demanding. In this work, we propose an approach to feature construction that is based on Meta Learning. Learning tasks are stored together with a corresponding set of constructed features in a case base. This case base is then used to constraint and guide the feature construction for new tasks. Our proposed method consists essentially of a new representation model for learning tasks and a corresponding two step distance measure. Our approach is unique as it enables us to apply case based feature construction not only on a large scale, but also in distributed learning scenarios in which communication cost plays an important role. Using the two step process, the accuracy of recommendations can be increased while not loosing the benefits of efficiency. The theoretical results are also confirmed by experiments on both synthetical data and data obtained from a distributed learning scenario on audio data.