Separate-and-conquer Regression Technical Report TUD – KE – 2010 – 01

In this paper a rule learning algorithm for the prediction of numerical target variables is presented. It is based on the separate-and-conquer strategy and the classification phase is done by a decision list. A new splitpoint generation method is introduced for the efficient handling of numerical attributes. It is shown that the algorithm performs comparable to other regression algorithms where some of them are based on rules and some are not. Additionally a novel heuristic for evaluating the trade-off between consistency and generality of regression rules is introduced. This heuristic features a parameter to directly trade off the rules consistency and its generality. We present an optimal setting for this parameter based on optimizing it on several data sets. The algorithm features two additional parameters that are also tuned on the same datasets as the heuristics parameter. The evaluation part of the paper gives insights on results obtained on tuning datasets that were split into two folds of equal size. The algorithm was tuned on the first set of these split databases and is evaluated on the hold-out folds and vice versa yielding two configurations of the rule learner. These are also evaluated on 9 testing datasets that were not used during the optimization.