An iterated classification rule based on auxiliary pseudo-predictors

Abstract This article proposes a two-step iterative procedure to improve the misclassification error rate of an initial classification rule. The first step involves an iterative method for generating a sequence of classifiers from the initial one; this is based on the augmentation of the feature vector with some new pseudo-predictors. Unlike other components of the feature vector, these new pseudo-predictors tend to provide information primarily on the performance or correctness of the classifier itself. The second step of the proposed procedure “pools together” the classifiers constructed in step one in order to produce a new classifier which is far more effective (in an asymptotic sense) than the initial classifier. In addition to these results, a data-splitting approach for selecting the number of iterations will also be discussed. Both the mechanics and the asymptotic validity of the proposed procedure are studied.