Evaluation of Machine-Learning Algorithm Ranking Advisors

Selecting the best-suited machine-learning classi cation algorithms for data-mining tasks is a meta-learning activity of the METAL project. Two alternative approaches for providing a user with an `advisor' a ranked preference list of algorithms have been implemented. In this paper results of experiments designed to test thoroughly the consistency of these models are presented. After considering a wide-view of how di erent user-pro les arise, an impartial methodology for independently comparing ranking advice is presented, which also enables overall advisor performance under two speci c user-preferences advice-horizon and time-value of advice to be distinguished.