Representation of functional data in neural networks

Functional data analysis (FDA) is an extension of traditional data analysis to functional data, for example spectra, temporal series, spatio-temporal images, gesture recognition data, etc. Functional data are rarely known in practice; usually a regular or irregular sampling is known. For this reason, some processing is needed in order to benefit from the smooth character of functional data in the analysis methods. This paper shows how to extend the radial-basis function networks (RBFN) and multi-layer perceptron (MLP) models to functional data inputs, in particular when the latter are known through lists of input-output pairs. Various possibilities for functional processing are discussed, including the projection on smooth bases, functional principal component analysis, functional centering and reduction, and the use of differential operators. It is shown how to incorporate these functional processing into the RBFN and MLP models. The functional approach is illustrated on a benchmark of spectrometric data analysis.

[1]  Maxwell B. Stinchcombe,et al.  Neural network approximation of continuous functionals and continuous functions on compactifications , 1999, Neural Networks.

[2]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[3]  I. Sandberg,et al.  Network approximation of input-output maps and functionals , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[4]  Hans Henrik Thodberg,et al.  A review of Bayesian neural networks with an application to near infrared spectroscopy , 1996, IEEE Trans. Neural Networks.

[5]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[6]  J. Ramsay,et al.  Principal components analysis of sampled functions , 1986 .

[7]  Frédéric Ferraty,et al.  Curves discrimination: a nonparametric functional approach , 2003, Comput. Stat. Data Anal..

[8]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[9]  J. O. Ramsay,et al.  Functional Data Analysis (Springer Series in Statistics) , 1997 .

[10]  Philippe Besse,et al.  Simultaneous non-parametric regressions of unbalanced longitudinal data , 1997 .

[11]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[12]  Les A. Piegl,et al.  Fundamental developments of computer-aided geometric modeling , 1993 .

[13]  Fabrice Rossi,et al.  Functional preprocessing for multilayer perceptrons , 2004, ESANN.

[14]  Frédéric Ferraty,et al.  The Functional Nonparametric Model and Application to Spectrometric Data , 2002, Comput. Stat..

[15]  Fabrice Rossi,et al.  Multi-layer Perceptrons for Functional Data Analysis: A Projection Based Approach , 2002, ICANN.

[16]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[17]  Fabrice Rossi,et al.  Functional multi-layer perceptron: a non-linear tool for functional data analysis , 2007, Neural Networks.

[18]  C. Abraham,et al.  Unsupervised Curve Clustering using B‐Splines , 2003 .

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[21]  Michel Verleysen,et al.  Functional radial basis function networks , 2004, ESANN.

[22]  L. Ferré,et al.  Functional sliced inverse regression analysis , 2003 .

[23]  H. H. Thodberg,et al.  Optimal minimal neural interpretation of spectra , 1992 .

[24]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[25]  François Fleuret,et al.  Theoretical properties of functional Multi Layer Perceptrons , 2002, ESANN.

[26]  I. Sandberg.  Notes on weighted norms and network approximation of functionals , 1996 .

[27]  Fabrice Rossi,et al.  Clustering functional data with the SOM algorithm , 2004, ESANN.

[28]  Catherine A. Sugar,et al.  Clustering for Sparsely Sampled Functional Data , 2003 .

[29]  J. Dauxois,et al.  Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference , 1982 .