ISUP Vol 52-Fasc 1-2

Let X be a random variable taking values in a Hilbert space and let Y be a random label with values in {0, 1}. Given a collection of classification rules and a learning sample of independent copies of the pair (X, Y ), it is shown how to select optimally and consistently a classifier. As a general strategy, the learning sample observations are first expanded on a wavelet basis and the overall infinite dimension is reduced to a finite one via a suitable data-dependent thresholding. Then, a finite-dimensional classification rule is performed on the non-zero coefficients. Both the dimension and the classifier are automatically selected by data-splitting and empirical risk minimization. Applications of this technique to a signal discrimination problem involving speech recordings and simulated data are presented.

[1]  Florentina Bunea,et al.  Functional classification in Hilbert spaces , 2005, IEEE Transactions on Information Theory.

[2]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[3]  Peter Hall,et al.  A Functional Data—Analytic Approach to Signal Discrimination , 2001, Technometrics.

[4]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[5]  Gunnar Rätsch,et al.  Advanced Lectures on Machine Learning , 2004, Lecture Notes in Computer Science.

[6]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[7]  Sanjeev R. Kulkarni,et al.  Rates of convergence of nearest neighbor estimation under arbitrary sampling , 1995, IEEE Trans. Inf. Theory.

[8]  Henry W. Altland,et al.  Applied Functional Data Analysis , 2003, Technometrics.

[9]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[10]  H. Cardot,et al.  Estimation in generalized linear models for functional data via penalized likelihood , 2005 .

[11]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[12]  Ricardo Fraiman,et al.  Robust estimation and classification for functional data via projection-based depth notions , 2007, Comput. Stat..

[13]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Gérard Biau,et al.  On the Kernel Rule for Function Classification , 2006 .

[15]  Frédéric Ferraty,et al.  The Functional Nonparametric Model and Application to Spectrometric Data , 2002, Comput. Stat..

[16]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[17]  Frédéric Ferraty,et al.  Curves discrimination: a nonparametric functional approach , 2003, Comput. Stat. Data Anal..

[18]  Gilbert Saporta,et al.  PLS regression on a stochastic process , 2001, Comput. Stat. Data Anal..

[19]  A. Haar.  Zur Theorie der orthogonalen Funktionensysteme , 1910 .

[20]  Arnaud Guyader,et al.  Nearest neighbor classification in infinite dimension , 2006 .

[21]  Gilbert Saporta,et al.  PLS classification of functional data , 2005, Comput. Stat..

[22]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[23]  Fabrice Rossi,et al.  Support Vector Machine For Functional Data Classification , 2006, ESANN.

[24]  S. Boucheron,et al.  Theory of classification : a survey of some recent advances , 2005 .

[25]  Anestis Antoniadis,et al.  Dimension reduction in functional regression with applications , 2006, Comput. Stat. Data Anal..

[26]  Luc Devroye,et al.  Automatic Pattern Recognition: A Study of the Probability of Error , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Magalie Fromont,et al.  Functional Classification with Margin Conditions , 2006, COLT.

[28]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .