Kernel Change-point Analysis

We introduce a kernel-based method for change-point analysis within a sequence of temporal observations. Change-point analysis of an unlabelled sample of observations consists in, first, testing whether a change in the distribution occurs within the sample, and second, if a change occurs, estimating the change-point instant after which the distribution of the observations switches from one distribution to another different distribution. We propose a test statistic based upon the maximum kernel Fisher discriminant ratio as a measure of homogeneity between segments. We derive its limiting distribution under the null hypothesis (no change occurs), and establish the consistency under the alternative hypothesis (a change occurs). This allows to build a statistical hypothesis testing procedure for testing the presence of a change-point, with a prescribed false-alarm probability and detection probability tending to one in the large-sample setting. If a change actually occurs, the test statistic also yields an estimator of the change-point location. Promising experimental results in temporal segmentation of mental tasks from BCI data and pop song indexation are presented.

[1]  Arjun K. Gupta,et al.  Parametric Statistical Change Point Analysis , 2000 .

[2]  Don R. Hush,et al.  An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels , 2006, IEEE Transactions on Information Theory.

[3]  J. Hammersley,et al.  Monte Carlo Methods , 1965 .

[4]  Kenji Fukumizu,et al.  Statistical Consistency of Kernel Canonical Correlation Analysis , 2007 .

[5]  Haikady N. Nagaraja,et al.  Inference in Hidden Markov Models , 2006, Technometrics.

[6]  Zaïd Harchaoui,et al.  A regularized kernel-based approach to unsupervised audio segmentation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Michèle Basseville,et al.  Detection of abrupt changes , 1993 .

[8]  Gilles Blanchard,et al.  Statistical properties of kernel principal component analysis , 2007, Machine Learning.

[9]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[10]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[11]  Fernando De la Torre,et al.  Temporal Segmentation of Facial Behavior , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  T. Lai.  SEQUENTIAL ANALYSIS: SOME CLASSICAL PROBLEMS AND NEW CHALLENGES , 2001 .

[13]  Chong Gu.  Smoothing Spline Anova Models , 2002 .

[14]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[15]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[16]  Gilles Blanchard,et al.  Finite-Dimensional Projection for Classification and Statistical Learning , 2008 .

[17]  Bernhard Schölkopf,et al.  Injective Hilbert Space Embeddings of Probability Measures , 2008, COLT.

[18]  A. Neath.  Testing Statistical Hypotheses (3rd ed.). E. L. Lehmann and Joseph P. Romano , 2006 .

[19]  O. Cappé,et al.  Retrospective Mutiple Change-Point Estimation with Kernels , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[20]  Manuel Davy,et al.  An online kernel change detection algorithm , 2005, IEEE Transactions on Signal Processing.

[21]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[22]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[23]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[24]  Eric Moulines,et al.  Inference in Hidden Markov Models (Springer Series in Statistics) , 2005 .

[25]  Zaïd Harchaoui,et al.  Testing for Homogeneity with Kernel Fisher Discriminant Analysis , 2007, NIPS.

[26]  D. Siegmund,et al.  Tests for a change-point , 1987 .

[27]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[28]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[29]  L. Horváth,et al.  Limit Theorems in Change-Point Analysis , 1997 .