Function approximation on non-Euclidean spaces

This paper presents a family of layered feed-forward networks that is able to uniformly approximate functions on any metric space, and also on a wide variety of non-metric spaces. Non-Euclidean input spaces are frequently encountered in practice, while usual approximation schemes are guaranteed to work only on Euclidean metric spaces. Theoretical foundations are provided, as well as practical algorithms and illustrative examples. This tool potentially constitutes a significant extension of the common notion of 'universal approximation capability'.

[1]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[2]  Robert A. Wagner,et al.  An Extension of the String-to-String Correction Problem , 1975, JACM.

[3]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[4]  Jungho Yoon,et al.  Interpolation by Radial Basis Functions on Sobolev Space , 2001, J. Approx. Theory.

[5]  T. Poggio,et al.  Networks and the best approximation property , 1990, Biological Cybernetics.

[6]  Pierre Courrieu,et al.  Straight monotonic embedding of data sets in Euclidean spaces , 2002, Neural Networks.

[7]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[8]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[9]  Pierre Courrieu.  A convergent generator of neural networks , 1993, Neural Networks.

[10]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[11]  P. G. Ciarlet,et al.  Introduction a l'analyse numerique matricielle et a l'optimisation , 1984 .

[12]  Vladik Kreinovich,et al.  Arbitrary nonlinearity is sufficient to represent all functions by neural networks: A theorem , 1991, Neural Networks.

[13]  Michael J. Fischer,et al.  The String-to-String Correction Problem , 1974, JACM.

[14]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[15]  A WagnerRobert,et al.  An Extension of the String-to-String Correction Problem , 1975 .

[16]  M. Okochi,et al.  Trapezoidal DP matching with time reversibility , 1982, ICASSP.

[17]  Bruce E. Rosen,et al.  Genetic Algorithms and Very Fast Simulated Reannealing: A comparison , 1992 .

[18]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[19]  Jonathan Grainger,et al.  Inverse discrimination time as a perceptual distance for alphabetic characters , 2004 .

[20]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[21]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[22]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[23]  Pierre Courrieu,et al.  The Hyperbell Algorithm for Global Optimization: A Random Walk Using Cauchy Densities , 1997, J. Glob. Optim..

[24]  Marcello Pelillo.  A relaxation algorithm for estimating the domain of validity of feedforward neural networks , 2004, Neural Processing Letters.

[25]  Jon G. Rokne,et al.  Computer Methods for the Range of Functions , 1984 .

[26]  A. A. Zhigli︠a︡vskiĭ,et al.  Theory of Global Random Search , 1991 .

[27]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[28]  C. Micchelli.  Interpolation of scattered data: Distance matrices and conditionally positive definite functions , 1986 .

[29]  Pierre Courrieu,et al.  Two methods for encoding clusters , 2001, Neural Networks.

[30]  Pierre Courrieu,et al.  Three algorithms for estimating the domain of validity of feedforward neural networks , 1994, Neural Networks.