Evaluating random forest models for irace

Automatic algorithm configurators can greatly improve the performance of algorithms by effectively searching the parameter space. As algorithm configuration tasks can have large parameter spaces and the execution of candidate algorithm configurations is often very costly in terms of computation time, further improvements in the search techniques used by automatic configurators are important and increase the applicability of available configuration methods. One common technique to improve the behavior of search methods when evaluations are computationally expensive are surrogate model techniques. These models are able to exploit the scarce available data and help to direct the search towards evaluating the most promising candidate configurations. In this paper, we study the use of random forests models as surrogate models in irace, a flexible automatic configuration tool based on iterated racing that has been successfully applied in the literature. We evaluate the performance of the random forest model using different settings when trained with data obtained from the irace configuration process and we evaluate their performance under similar conditions as in the configuration process. This preliminary work aims at providing guidelines for the incorporation of random forest to the configuration process of irace.