Constructive Approximation of Discontinuous Functions by Neural Networks

In this paper, we give a constructive proof that a real, piecewise continuous function can be almost uniformly approximated by single hidden-layer feedforward neural networks (SLFNNs). The construction procedure avoids the Gibbs phenomenon. Computer experiments show that the resulting approximant is much more accurate than SLFNNs trained by gradient descent.

[1]  Wei Cai,et al.  Essentially Nonoscillatory Spectral Fourier Method for Shocks Wave Calculations , 1989 .

[2]  Edwin Hewitt,et al.  Real And Abstract Analysis , 1967 .

[3]  J.L.F. Muniz.  Qualitative Korovkin-Type Theorems for RF-Convergence , 1995 .

[4]  Alvaro R. De Pierro,et al.  Detection of edges from spectral data: New results , 2007 .

[5]  Frank L. Lewis,et al.  Neural network approximation of piecewise continuous functions: application to friction compensation , 1997, Proceedings of 12th IEEE International Symposium on Intelligent Control.

[6]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[7]  S. Mallat.  Multiresolution approximations and wavelet orthonormal bases of L^2(R) , 1989 .

[8]  J. Foster,et al.  The Gibbs phenomenon for piecewise-linear approximation , 1991 .

[9]  J. Joseph,et al.  Fourier Series , 2018, Series and Products in the Development of Mathematics.

[10]  A. Friedman.  Foundations of modern analysis , 1970 .

[11]  Richard G. Baraniuk,et al.  Nonlinear wavelet transforms for image coding via lifting , 2003, IEEE Trans. Image Process..

[12]  Chi-Wang Shu,et al.  On the Gibbs Phenomenon and Its Resolution , 1997, SIAM Rev..

[13]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[14]  Anne Gelb,et al.  Detection of Edges in Spectral Data , 1999 .

[15]  J. Foster,et al.  Gibbs-Wilbraham splines , 1995 .

[16]  H. Weyl,et al.  Die gibbs’sche erscheinung in der theorie der kugelfunktionen , 1910 .

[17]  Hans Volkmer,et al.  Gibbs' phenomenon in higher dimensions , 2007, J. Approx. Theory.

[18]  Stephen P. Boyd,et al.  A bisection method for computing the H∞ norm of a transfer matrix and related problems , 1989, Math. Control. Signals Syst..

[19]  F. J. Sainz,et al.  Constructive approximate interpolation by neural networks , 2006 .

[20]  Frank L. Lewis,et al.  Deadzone compensation in motion control systems using neural networks , 2000, IEEE Trans. Autom. Control..

[21]  Robert M. Burton,et al.  Universal approximation in p-mean by neural networks , 1998, Neural Networks.

[22]  J. Gibbs.  Fourier's Series , 1898, Nature.

[23]  S. E. Kelly,et al.  Gibbs Phenomenon for Wavelets , 1996 .

[24]  Tobin A. Driscoll,et al.  A Padé-based algorithm for overcoming the Gibbs phenomenon , 2004, Numerical Algorithms.

[25]  E. Candès,et al.  Ridgelets: a key to higher-dimensional intermittency? , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[26]  G. Tolstov.  Fourier Series , 1962 .

[27]  Abdul J. Jerri.  The Gibbs Phenomenon in Fourier Analysis, Splines and Wavelet Approximations , 1998 .

[28]  D. Donoho.  Wedgelets: nearly minimax estimation of edges , 1999 .

[29]  Leonardo Colzani,et al.  The Gibbs Phenomenon for Multiple Fourier Integrals , 1995 .

[30]  Tomoki Fukai,et al.  A model cortical circuit for the storage of temporal sequences , 1995, Biological Cybernetics.

[31]  Jan S. Hesthaven,et al.  Padé-Legendre Interpolants for Gibbs Reconstruction , 2006, J. Sci. Comput..

[32]  Gilbert Helmberg.  A corner point Gibbs phenomenon for Fourier series in two dimensions , 1999 .

[33]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[34]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[35]  Wei Cai,et al.  Essentially Nonoscillatory Spectral Fourier Method for Shocks Wave Calculations , 1989 .

[36]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .