On the Kernel Rule for Function Classification

AbstractLet X be a random variable taking values in a function space $$\mathcal{F}$$, and let Y be a discrete random label with values 0 and 1. We investigate asymptotic properties of the moving window classification rule based on independent copies of the pair (X,Y). Contrary to the finite dimensional case, it is shown that the moving window classifier is not universally consistent in the sense that its probability of error may not converge to the Bayes risk for some distributions of (X,Y). Sufficient conditions both on the space $$\mathcal{F}$$ and the distribution of X are then given to ensure consistency.

[1]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[2]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[3]  Anestis Antoniadis,et al.  Wavelet methods for continuous-time prediction using Hilbert-valued autoregressive processes , 2003 .

[4]  Frédéric Ferraty,et al.  Curves discrimination: a nonparametric functional approach , 2003, Comput. Stat. Data Anal..

[5]  Peter Hall,et al.  A Functional Data—Analytic Approach to Signal Discrimination , 2001, Technometrics.

[6]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[7]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[8]  W. Rudin.  Real and complex analysis , 1968 .

[9]  Sanjeev R. Kulkarni,et al.  Rates of convergence of nearest neighbor estimation under arbitrary sampling , 1995, IEEE Trans. Inf. Theory.

[10]  Pertti Mattila.  Differentiation of measures on uniform spaces , 1980 .

[11]  J. Siemons.  Surveys in combinatorics, 1989 , 1989 .

[12]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[13]  B. Silverman,et al.  Estimating the mean and covariance structure nonparametrically when the data are curves , 1991 .

[14]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[15]  L. Devroye,et al.  An equivalence theorem for L1 convergence of the kernel regression estimate , 1989 .

[16]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[17]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[18]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[19]  Florentina Bunea,et al.  Functional classification in Hilbert spaces , 2005, IEEE Transactions on Information Theory.

[20]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[21]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[22]  J. Tiser.  Differentiation theorem for Gaussian measures on Hilbert space , 1988 .

[23]  E. Nadaraya.  On Estimating Regression , 1964 .

[24]  Z. Ciesielski.  Hölder conditions for realizations of Gaussian processes , 1961 .

[25]  T. Gasser,et al.  Statistical Tools to Analyze Data Representing a Sample of Curves , 1992 .

[26]  S. Boucheron,et al.  Theory of classification : a survey of some recent advances , 2005 .

[27]  Frédéric Ferraty,et al.  Dimension fractale et estimation de la régression dans des espaces vectoriels semi-normés , 2000 .

[28]  Frédéric Ferraty,et al.  The Functional Nonparametric Model and Application to Spectrometric Data , 2002, Comput. Stat..

[29]  E. Nadaraya.  Remarks on Non-Parametric Estimates for Density Functions and Regression Curves , 1970 .

[30]  H. Akaike.  An approximation to the density function , 1954 .

[31]  David Preiss,et al.  Differentiation of measures on Hilbert spaces , 1982 .

[32]  D. Bosq.  Linear Processes in Function Spaces: Theory And Applications , 2000 .

[33]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .