New Error Bounds for Deep ReLU Networks Using Sparse Grids

We prove a theorem concerning the approximation of multivariate functions by deep ReLU networks. We present new error estimates for which the curse of dimensionality is lessened by establishing a c...

[1]  Jie Shen,et al.  Sparse Spectral Approximations of High-Dimensional Problems Based on Hyperbolic Cross , 2010, SIAM J. Numer. Anal..

[2]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[3]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[4]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[5]  E. N. Oberg.  The approximate solution of integral equations , 1935 .

[6]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[7]  M. Irani.  Vision Day Schedule Time Speaker and Collaborators Affiliation Title a General Preprocessing Method for Improved Performance of Epipolar Geometry Estimation Algorithms on the Expressive Power of Deep Learning: a Tensor Analysis , 2016 .

[8]  Winfried Sickel,et al.  Spaces of functions of mixed smoothness and approximation from hyperbolic crosses , 2004, J. Approx. Theory.

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  Lorenzo Rosasco,et al.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review , 2016, International Journal of Automation and Computing.

[11]  A. Grossmann,et al.  DECOMPOSITION OF HARDY FUNCTIONS INTO SQUARE INTEGRABLE WAVELETS OF CONSTANT SHAPE , 1984 .

[12]  Frauke Sprengel,et al.  Periodic interpolation and wavelets on sparse grids , 1998, Numerical Algorithms.

[13]  Philipp Petersen,et al.  Optimal approximation of piecewise smooth functions using deep ReLU neural networks , 2017, Neural Networks.

[14]  Matus Telgarsky,et al.  Benefits of Depth in Neural Networks , 2016, COLT.

[15]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[16]  R. Srikant,et al.  Why Deep Neural Networks for Function Approximation? , 2016, ICLR.

[17]  H. N. Mhaskar,et al.  Neural Networks for Optimal Approximation of Smooth and Analytic Functions , 1996, Neural Computation.

[18]  Dmitry Yarotsky,et al.  Error bounds for approximations with deep ReLU networks , 2016, Neural Networks.

[19]  Ohad Shamir,et al.  The Power of Depth for Feedforward Neural Networks , 2015, COLT.