Generalization Error Bounds for Aggregate Classifiers

The use of multiple classifiers has raised much interest in the statistical learning community in the past few years. The basic principle of multiple classifiers algorithms, also called aggregation or ensemble or voting methods, is to construct, according to some algorithm, several (generally a few dozens) different classifiers belonging to a certain family (e.g. support vector machines, classification trees, neural nets...). The “aggregate” classifier is then obtained by majority vote among the outputs of the single constructed classifiers once they are presented a new instance. For some algorithms the majority vote is replaced by a weighted vote, with weights prescribed by the aggregation algorithm. Classical references about this kind of methods include [2, 3, 9, 8, 12, 14, 19].