Investigating Learning Behavior with Experiment Databases

Gaining insights into the behavior of learning algorithms generally involves studying their performance on many dierent datasets, under various parameter settings. A useful method to learn eectively from previous learning episodes are experiment databases: databases designed to store detailed descriptions of a large number of learning experiments, including the used algorithms, parameter settings, datasets (and preprocessing techniques), evaluation environment, and a large number of measured performance criteria. The experiments themselves are selected to cover a wide range of conditions. After being populated, these databases allow us to investigate a wide range of questions on algorithm behavior by just querying the database and interpreting the returned results, or by using data mining methods to automatically discover patterns in learning algorithm performance. For instance, one could compare or rank algorithms by querying for