Boosted decision graphs for NLP learning tasks

This paper reports the implementation of DRAPH-GP an extension of the decision graph algorithm DGRAPH-OW using the AdaBoost algorithm. This algorithm, which we call 1-Stage Boosting, is shown to improve the accuracy of decision graphs, along with another technique which we combine with AdaBoost and call 2-Stage Boosting which shows greater improvement. Empirical tests demonstrate that both 1-Stage and 2-Stage Boosting techniques perform better than the boosted C4.5 algorithm (C5.0). The boosting has shown itself competitive for NLP tasks with a high disjunction of attribute space against memory based methods, and potentially better if part of an Hierarchical Multi-Method Classifier. An explanation for the effectiveness of boosting due to a poor choice of prior probabilities is presented. 1. INTRODUCTION In a wide variety of classification problems, boosting techniques have proven to be an effective method to significantly reduce the error of weak learning algorithms. While the AdaBoost algorithm (Freund & Schapire, 1995) has been used to improve the accuracy of a decision tree algorithm (Quinlan & Rivest, 1989), which uses the Minimum Description Length Principle (MDL), little is known about it's effectiveness on the decision graphs.