FUNCTIONAL SUPERVISED CLASSIFICATION WITH WAVELETS

Let X be a random variable taking values in a Hilbert space and let Y be a random label with values in {0,1}. Given a collection of classification rules and a learning sample of independent, copies of the pair (X,Y), it is shown how to select optimally and consistently a classifier. As a general strategy, the learning sample observations are first expanded on a wavelet basis and the overall infinite dimension is reduced to a finite one via a suitable data-dependent thresholding. Then, a finite-dimensional classification rule is performed on the non-zero coefficients. Both the dimension and the classifier are automatically selected by data-splitting and empirical risk minimization. Applications of this technique to a signal discrimination problem involving speech recordings and simulated data are presented.

[1]  James O. Ramsay,et al.  Applied Functional Data Analysis: Methods and Case Studies , 2002 .

[2]  A. Haar.  Zur Theorie der orthogonalen Funktionensysteme , 1910 .

[3]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[4]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[5]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[6]  Arnaud Guyader,et al.  Nearest neighbor classification in infinite dimension , 2006 .

[7]  Anestis Antoniadis,et al.  Dimension reduction in functional regression with applications , 2006, Comput. Stat. Data Anal..

[8]  Sanjeev R. Kulkarni,et al.  Rates of convergence of nearest neighbor estimation under arbitrary sampling , 1995, IEEE Trans. Inf. Theory.

[9]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Magalie Fromont,et al.  Functional Classification with Margin Conditions , 2006, COLT.

[11]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[12]  S. Boucheron,et al.  Theory of classification : a survey of some recent advances , 2005 .

[13]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[14]  Gérard Biau,et al.  On the Kernel Rule for Function Classification , 2006 .

[15]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[16]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[17]  Gilbert Saporta,et al.  PLS classification of functional data , 2005, Comput. Stat..

[18]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[19]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[20]  Ricardo Fraiman,et al.  Robust estimation and classification for functional data via projection-based depth notions , 2007, Comput. Stat..

[21]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[22]  Fabrice Rossi,et al.  Support Vector Machine For Functional Data Classification , 2006, ESANN.

[23]  H. Cardot,et al.  Estimation in generalized linear models for functional data via penalized likelihood , 2005 .

[24]  Luc Devroye,et al.  Automatic Pattern Recognition: A Study of the Probability of Error , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Frédéric Ferraty,et al.  The Functional Nonparametric Model and Application to Spectrometric Data , 2002, Comput. Stat..

[26]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[27]  Florentina Bunea,et al.  Functional classification in Hilbert spaces , 2005, IEEE Transactions on Information Theory.

[28]  Frédéric Ferraty,et al.  Curves discrimination: a nonparametric functional approach , 2003, Comput. Stat. Data Anal..

[29]  Peter Hall,et al.  A Functional Data—Analytic Approach to Signal Discrimination , 2001, Technometrics.

[30]  Henry W. Altland,et al.  Applied Functional Data Analysis , 2003, Technometrics.

[31]  Gilbert Saporta,et al.  PLS regression on a stochastic process , 2001, Comput. Stat. Data Anal..

[32]  Gunnar Rätsch,et al.  Advanced Lectures on Machine Learning , 2004, Lecture Notes in Computer Science.

[33]  Z. Q. John Lu,et al.  Nonparametric Functional Data Analysis: Theory And Practice , 2007, Technometrics.