A data-dependent skeleton estimate for learning

We introduce and analyze a method for learning based on minimizing the empirical risk over a carefully selected “skeleton” of a class of regression functions (concepts). The skeleton is a covering of the class based on a datadependent metric, especially fitted for classification. The method is shown to outperform empirical risk minimization and other non-datadependent skeleton-based estimates.