论文信息 - Oracle Bounds and Exact Algorithm for Dyadic Classification Trees

Oracle Bounds and Exact Algorithm for Dyadic Classification Trees

This paper introduces a new method using dyadic decision trees for estimating a classification or a regression function in a multiclass classification problem. The estimator is based on model selection by penalized empirical loss minimization. Our work consists in two complementary parts: first, a theoretical analysis of the method leads to deriving oracle-type inequalities for three different possible loss functions. Secondly, we present an algorithm able to compute the estimator in an exact way.

[1] P. Massart,et al. Risk bounds for model selection via penalization , 1999 .

[2] P. Massart. Some applications of concentration inequalities to statistics , 2000 .

[3] M. AdelsonVelskii,et al. AN ALGORITHM FOR THE ORGANIZATION OF INFORMATION , 1963 .

[4] Robert D. Nowak,et al. Near-Minimax Optimal Classification with Dyadic Classification Trees , 2003, NIPS.

[5] A. Tsybakov,et al. Optimal aggregation of classifiers in statistical learning , 2003 .

[6] G. Castellan. Sélection d'histogrammes à l'aide d'un critère de type Akaike , 2000 .

[7] Gilles Blanchard,et al. On the Rate of Convergence of Regularized Boosting Classifiers , 2003, J. Mach. Learn. Res..

[8] Servane Gey,et al. Risk Bounds for CART Regression Trees , 2003 .

[9] D. Donoho. CART AND BEST-ORTHO-BASIS: A CONNECTION' , 1997 .

[10] Robert D. Nowak,et al. Dyadic Classification Trees via Structural Risk Minimization , 2002, NIPS.

[11] László Györfi,et al. A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[12] David D. Denison,et al. Nonlinear estimation and classification , 2003 .

[13] P. Massart,et al. Risk bounds for statistical learning , 2007, math/0702683.

[14] A. Barron,et al. APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[15] Leo Breiman,et al. Classification and Regression Trees , 1984 .