The recovery of ridge functions on the hypercube suffers from the curse of dimensionality

A multivariate ridge function is a function of the form $f(x) = g(a^{\scriptscriptstyle T} x)$, where $g$ is univariate and $a \in \mathbb{R}^d$. We show that the recovery of an unknown ridge function defined on the hypercube $[-1,1]^d$ with Lipschitz-regular profile $g$ suffers from the curse of dimensionality when the recovery error is measured in the $L_\infty$-norm, even if we allow randomized algorithms. If a limited number of components of $a$ is substantially larger than the others, then the curse of dimensionality is not present and the problem is weakly tractable provided the profile $g$ is sufficiently regular.

[1]  Henryk Wozniakowski,et al.  Approximation of infinitely differentiable multivariate functions is intractable , 2009, J. Complex..

[2]  Nikolai Sergeevich Bakhvalov,et al.  On the approximate calculation of multiple integrals , 2015, J. Complex..

[3]  H. Ichimura.  Estimation of single index models , 1987 .

[4]  W. Härdle.  Nonparametric and Semiparametric Models , 2004 .

[5]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[6]  A. Pinkus,et al.  Identifying Linear Combinations of Ridge Functions , 1999 .

[7]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[8]  Wolfgang Dahmen,et al.  Approximation of High-Dimensional Rank One Tensors , 2013, Constructive Approximation.

[9]  David L. Donoho,et al.  Orthonormal Ridgelets and Linear Singularities , 2000, SIAM J. Math. Anal..

[10]  E. Novak,et al.  Tractability of Multivariate Problems Volume II: Standard Information for Functionals , 2010 .

[11]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[12]  J. Spencer.  Probabilistic Methods in Combinatorics , 1974 .

[13]  Markus Weimar,et al.  Notes on (s, t)-weak tractability: A refined classification of problems with (sub)exponential information complexity , 2014, J. Approx. Theory.

[14]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[15]  Allan Pinkus.  Approximating by Ridge Functions , 1997 .

[16]  A. Juditsky,et al.  Direct estimation of the index coefficient in a single-index model , 2001 .

[17]  Erich Novak,et al.  Tractability of the Approximation of High-Dimensional Rank One Tensors , 2014, Constructive Approximation.

[18]  Jan Vybíral,et al.  Entropy and Sampling Numbers of Classes of Ridge Functions , 2013, 1311.2005.

[19]  Holger Rauhut,et al.  An Invitation to Compressive Sensing , 2013 .

[20]  W. Härdle,et al.  Optimal Smoothing in Single-index Models , 1993 .

[21]  E. Candès.  Harmonic Analysis of Neural Networks , 1999 .

[22]  Po-Shen Loh,et al.  Probabilistic Methods in Combinatorics , 2009 .

[23]  P. J. Huber.  Projection Pursuit for , 2022 .

[24]  Jan Vybíral,et al.  On some aspects of approximation of ridge functions , 2014, J. Approx. Theory.

[25]  Thomas M. Stoker.  Consistent estimation of scaled coefficients , 2011 .

[26]  Henryk Wozniakowski,et al.  Quasi-polynomial tractability , 2011, J. Complex..

[27]  E. Novak,et al.  Tractability of Multivariate Problems , 2008 .

[28]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[29]  Jan Vybíral,et al.  Learning Functions of Few Arbitrary Linear Parameters in High Dimensions , 2010, Found. Comput. Math..

[30]  P. Hall.  On Projection Pursuit Regression , 1989 .

[31]  V. Cevher,et al.  Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods , 2013, 1310.1826.

[32]  Pawel Siedlecki,et al.  Uniform weak tractability , 2013, J. Complex..

[33]  E. Novak,et al.  Tractability of Multivariate Problems, Volume III: Standard Information for Operators. , 2012 .

[34]  Lixing Zhu,et al.  The EFM approach for single-index models , 2011, 1211.5220.

[35]  Stefan Heinrich,et al.  Lower bounds for the complexity of Monte Carlo function approximation , 1992, J. Complex..

[36]  E. Candès.  Ridgelets: estimating with ridge functions , 2003 .

[37]  Martin J. Wainwright,et al.  Minimax-Optimal Rates For Sparse Additive Models Over Kernel Classes Via Convex Programming , 2010, J. Mach. Learn. Res..

[38]  I. Daubechies,et al.  Capturing Ridge Functions in High Dimensions from Point Queries , 2012 .

[39]  Thomas M. Stoker,et al.  Semiparametric Estimation of Index Coefficients , 1989 .