Complexity estimates based on integral transforms induced by computational units

Integral transforms with kernels corresponding to computational units are exploited to derive estimates of network complexity. The estimates are obtained by combining tools from nonlinear approximation theory and functional analysis together with representations of functions in the form of infinite neural networks. The results are applied to perceptron networks.

[1]  Vera Kurková.  Model Complexity of Neural Networks and Integral Transforms , 2009, ICANN.

[2]  Kevin Warwick,et al.  Computer Intensive Methods in Control and Signal Processing: The Curse of Dimensionality , 1997 .

[3]  Holger Wendland,et al.  Kernel techniques: From machine learning to meshless methods , 2006, Acta Numerica.

[4]  Vladik Kreinovich,et al.  Estimates of the Number of Hidden Units and Variation with Respect to Half-Spaces , 1997, Neural Networks.

[5]  Marcello Sanguineti,et al.  Geometric Upper Bounds on Rates of Variable-Basis Approximation , 2008, IEEE Transactions on Information Theory.

[6]  Yoshifusa Ito,et al.  Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory , 1991, Neural Networks.

[7]  C. Darken,et al.  Constructive Approximation Rates of Convex Approximation in Non-hilbert Spaces , 2022 .

[8]  Leonid Gurvits,et al.  Approximation and Learning of Convex Superpositions , 1997, J. Comput. Syst. Sci..

[9]  Paul C. Kainen,et al.  An Integral Upper Bound for Neural Network Approximation , 2009, Neural Computation.

[10]  Vera Kurková,et al.  Rates of approximation of real-valued boolean functions by neural networks , 1998, ESANN.

[11]  Eugene Lavretsky,et al.  On the geometric convergence of neural approximations , 2002, IEEE Trans. Neural Networks.

[12]  W. Rudin.  Real and complex analysis , 1968 .

[13]  Terrence L. Fine,et al.  Feedforward Neural Network Methodology , 1999, Information Science and Statistics.

[14]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[15]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[16]  Věra Kůrková,et al.  Dimension-Independent Rates of Approximation by Neural Networks , 1997 .

[17]  Marcello Sanguineti,et al.  Error Estimates for Approximate Optimization by the Extended Ritz Method , 2005, SIAM J. Optim..

[18]  L. Ahlfors.  Complex Analysis , 1979 .

[19]  Marcello Sanguineti,et al.  Comparison of worst case errors in linear and neural network approximation , 2002, IEEE Trans. Inf. Theory.

[20]  Hrushikesh Narhar Mhaskar,et al.  On the tractability of multivariate integration and approximation by neural networks , 2004, J. Complex..

[21]  Paul C. Kainen,et al.  A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves , 2007, J. Approx. Theory.

[22]  G. Pisier.  Remarques sur un résultat non publié de B. Maurey , 1981 .

[23]  Eduardo D. Sontag,et al.  Rate of approximation results motivated by robust neural network learning , 1993, COLT '93.

[24]  Hrushikesh Narhar Mhaskar,et al.  Weighted quadrature formulas and approximation by zonal function networks on the sphere , 2006, J. Complex..

[25]  Lakhmi C. Jain,et al.  Handbook on Neural Information Processing , 2013, Handbook on Neural Information Processing.

[26]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[27]  Paul C. Kainen,et al.  Integral combinations of Heavisides , 2010 .

[28]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[29]  Marcello Sanguineti,et al.  Complexity of Gaussian-radial-basis networks approximating smooth functions , 2009, J. Complex..

[30]  A. Pietsch.  Eigenvalues and S-Numbers , 1987 .

[31]  Vera Kurková,et al.  Minimization of Error Functionals over Perceptron Networks , 2008, Neural Computation.