Near-optimal max-affine estimators for convex regression

This paper considers least squares estimators for regression problems over convex, uniformly bounded, uniformly Lipschitz function classes minimizing the empirical risk over max-affine functions (the maximum of finitely many affine functions). Based on new results on nonlinear nonparametric regression and on the approximation accuracy of maxaffine functions, these estimators are proved to achieve the optimal rate of convergence up to logarithmic factors. Preliminary experiments indicate that a simple randomized approximation to the optimal estimator is competitive with state-of-the-art alternatives.

[1]  E. M. Bronshteyn,et al.  The approximation of convex sets by polyhedra , 1975 .

[2]  E. Bronshtein ε-Entropy of convex sets and functions , 1976 .

[3]  Charles A. Holloway Technical Note - On the Estimation of Convex Functions , 1979, Oper. Res..

[4]  H. Varian The Nonparametric Approach to Demand Analysis , 1982 .

[5]  H. Varian The Nonparametric Approach to Production Analysis , 1984 .

[6]  D. Pollard Empirical Processes: Theory and Applications , 1990 .

[7]  Gábor Lugosi,et al.  Minimax regret under log loss for general classes of experts , 1999, COLT '99.

[8]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[9]  V. Buldygin,et al.  Metric characterization of random variables and random processes , 2000 .

[10]  A. W. van der Vaart,et al.  Uniform Central Limit Theorems , 2001 .

[11]  S. R. Jammalamadaka,et al.  Empirical Processes in M-Estimation , 2001 .

[12]  Robert Buff Continuous Time Finance , 2002 .

[13]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[14]  P. Bartlett,et al.  Local Rademacher complexities , 2005, math/0508275.

[15]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[16]  Timo Kuosmanen Representation Theorem for Convex Nonparametric Least Squares , 2008 .

[17]  Stephen P. Boyd,et al.  Convex piecewise-linear fitting , 2009 .

[18]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[19]  Ambuj Tewari,et al.  Smoothness, Low Noise and Fast Rates , 2010, NIPS.

[20]  E. Seijo,et al.  Nonparametric Least Squares Estimation of a Multivariate Convex Regression Function , 2010, 1003.4765.

[21]  Adityanand Guntuboyina Optimal rates of convergence for convex set estimation from support functions , 2011, 1108.5341.

[22]  V. Koltchinskii,et al.  Oracle inequalities in empirical risk minimization and sparse recovery problems , 2011 .

[23]  David B. Dunson,et al.  Ensemble Methods for Convex Regression with Applications to Geometric Programming Based Circuit Design , 2012, ICML.

[24]  Peter W. Glynn,et al.  Consistency of Multidimensional Convex Regression , 2012, Oper. Res..

[25]  Adityanand Guntuboyina,et al.  Covering Numbers for Convex Functions , 2012, IEEE Transactions on Information Theory.

[26]  Timo Kuosmanen,et al.  A more efficient algorithm for Convex Nonparametric Least Squares , 2013, Eur. J. Oper. Res..

[27]  S. Mendelson,et al.  Learning subgaussian classes : Upper and minimax bounds , 2013, 1305.4825.

[28]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[29]  David B. Dunson,et al.  Multivariate convex regression with adaptive partitioning , 2011, J. Mach. Learn. Res..

[30]  Csaba Szepesvári,et al.  A Finite-Sample Generalization Bound for Semiparametric Regression: Partially Linear Models , 2014, AISTATS.

[31]  Shahar Mendelson,et al.  Learning without Concentration , 2014, COLT.