Complexity regularization via localized random penalties

In this article, model selection via penalized empirical loss minimization in nonparametric classification problems is studied. Datadependent penalties are constructed, which are based on estimates of the complexity of a small subclass of each model class, containing only those functions with small empirical loss. The penalties are novel since those considered in the literature are typically based on the entire model class. Oracle inequalities using these penalties are established, and the advantage of the new penalties over those based on the complexity of the whole model class is demonstrated. 1. Introduction. In this article, we propose a new complexity-penalized model selection method based on data-dependent penalties. We consider the binary classification problem where, given a random observation X ∈ R d , one has to predict Y ∈ {0,1}. A classifier or classification rule is a function f : R d → {0,1}, with loss