Experiments with Projection Learning

Excessive information is known to degrade the classification performance of many machine learning algorithms. Attribute-efficient learning algorithms can tolerate irrelevant attributes without their performance being affected too much. Valiant's projection learning is a way to combine such algorithms so that this desired property is maintained. The archetype attribute-efficient learning algorithm Winnow and, especially, combinations of Winnow have turned out empirically successful in domains containing many attributes. However, projection learning as proposed by Valiant has not yet been evaluated empirically. We study how projection learning relates to using Winnow as such and with an extended set of attributes. We also compare projection learning with decision tree learning and Naive Bayes on UCI data sets.Projection learning systematically enhances the classification accuracy of Winnow, but the cost in time and space consumption can be high. Balanced Winnow seems to be a better alternative than the basic algorithm for learning the projection hypotheses. However, Balanced Winnow is not well suited for learning the second level (projective disjunction) hypothesis. The on-line approach projection learning does not fall far behind in classification accuracy from batch algorithms such as decision tree learning and Naive Bayes on the UCI data sets that we used.