Kernel change-point detection

We tackle the change-point problem with data belonging to a general set. We propose a penalty for choosing the number of change-points in the kernel-based method of Harchaoui and Cappe (2007). This penalty generalizes the one proposed for one dimensional signals by Lebarbier (2005). We prove it satisfies a non-asymptotic oracle inequality by showing a new concentration result in Hilbert spaces. Experiments on synthetic and real data illustrate the accuracy of our method, showing it can detect changes in the whole distribution of data, even when the mean and variance are constant. Our algorithm can also deal with data of complex nature, such as the GIST descriptors which are commonly used for video temporal segmentation.

[1]  David S. Matteson,et al.  ecp: An R Package for Nonparametric Multiple Change Point Analysis of Multivariate Data , 2013, 1309.3295.

[2]  Francis R. Bach,et al.  Learning Sparse Penalties for Change-point Detection using Max Margin Interval Regression , 2013, ICML.

[3]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[4]  Francis R. Bach,et al.  Large-Margin Metric Learning for Partitioning Problems , 2013, ArXiv.

[5]  Sivaraman Balakrishnan,et al.  Optimal kernel choice for large-scale two-sample tests , 2012, NIPS.

[6]  Hao Chen,et al.  Graph-based change-point detection , 2012, 1209.1625.

[7]  Eric P. Xing,et al.  Enabling dynamic network analysis through visualization in TVNViewer , 2012, BMC Bioinformatics.

[8]  Bertrand Michel,et al.  Slope heuristics: overview and implementation , 2011, Statistics and Computing.

[9]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[10]  P. Fearnhead,et al.  Optimal detection of changepoints with a linear computational cost , 2011, 1101.1438.

[11]  Jean-Marc Bardet,et al.  Multiple breaks detection in general causal time series using penalized quasi-likelihood , 2012 .

[12]  Stéphane Robin,et al.  Joint segmentation, calling, and normalization of multiple CGH profiles. , 2011, Biostatistics.

[13]  Jean-Philippe Vert,et al.  The group fused Lasso for multiple change-point detection , 2011, 1106.4199.

[14]  Cécile Ané,et al.  Detecting Phylogenetic Breakpoints and Discordance from Genome-Wide Alignments for Species Tree Reconstruction , 2011, Genome biology and evolution.

[15]  Olga Korosteleva,et al.  Mathematical Statistics: Asymptotic Minimax Theory , 2011 .

[16]  Sylvain Arlot,et al.  Segmentation of the mean of heteroscedastic data via cross-validation , 2009, Stat. Comput..

[17]  N. Akakpo.  Estimating a discrete distribution via histogram selection , 2011 .

[18]  Kenji Fukumizu,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2010, J. Mach. Learn. Res..

[19]  Ulrike von Luxburg,et al.  Clustering Stability: An Overview , 2010, Found. Trends Mach. Learn..

[20]  Piotr Kokoszka,et al.  Detecting changes in the mean of functional observations , 2009 .

[21]  Marie-Claude Sauvé,et al.  Histogram selection in non gaussian regression , 2009 .

[22]  V. Liebscher,et al.  Consistencies and rates of convergence of jump-penalized least squares estimators , 2009, 0902.4838.

[23]  Bernhard Schölkopf,et al.  Injective Hilbert Space Embeddings of Probability Measures , 2008, COLT.

[24]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[25]  Subhransu Maji,et al.  Classification using intersection kernel support vector machines is efficient , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Bernhard Schölkopf,et al.  Kernel Measures of Conditional Dependence , 2007, NIPS.

[27]  Ding-Xuan Zhou,et al.  Learning Theory: An Approximation Theory Viewpoint , 2007 .

[28]  P. Massart,et al.  Minimal Penalties for Gaussian Model Selection , 2007 .

[29]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[30]  Shai Ben-David,et al.  A Sober Look at Clustering Stability , 2006, COLT.

[31]  Gueorgi Kossinets,et al.  Empirical Analysis of an Evolving Social Network , 2006, Science.

[32]  Marc Lavielle,et al.  Using penalized contrasts for the change-point problem , 2005, Signal Process..

[33]  Emilie Lebarbier,et al.  Detecting multiple change-points in the mean of Gaussian process by model selection , 2005, Signal Process..

[34]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[35]  Matthias Hein,et al.  Hilbertian Metrics and Positive Definite Kernels on Probability Measures , 2005, AISTATS.

[36]  Michael I. Jordan,et al.  Dimensionality Reduction for Supervised Learning with Reproducing Kernel Hilbert Spaces , 2004, J. Mach. Learn. Res..

[37]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[38]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[39]  Michael I. Jordan,et al.  Kernel Dimensionality Reduction for Supervised Learning , 2003, NIPS.

[40]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[41]  P. Reynaud-Bouret,et al.  Exponential Inequalities, with Constants, for U-statistics of Order Two , 2003 .

[42]  Thomas Gärtner,et al.  Kernels for structured data , 2008, Series in Machine Perception and Artificial Intelligence.

[43]  Émilie Lebarbier,et al.  Quelques approches pour la détection de ruptures à horizon fini , 2002 .

[44]  O. Bousquet.  A Bennett concentration inequality and its application to suprema of empirical processes , 2002 .

[45]  P. Massart,et al.  Gaussian model selection , 2001 .

[46]  Irena Koprinska,et al.  Temporal video segmentation: A survey , 2001, Signal Process. Image Commun..

[47]  C. Mallows.  Some Comments on Cp , 2000, Technometrics.

[48]  É. Moulines,et al.  Least‐squares Estimation of an Unknown Number of Shifts in a Time Series , 2000 .

[49]  Irène Gijbels,et al.  On the Estimation of Jump Points in Smooth Curves , 1999 .

[50]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[51]  Edward Carlstein,et al.  Change-point problems , 1994 .

[52]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[53]  M. Wand,et al.  EXACT MEAN INTEGRATED SQUARED ERROR , 1992 .

[54]  Yi-Ching Yao.  Estimating the number of change-points via Schwarz' criterion , 1988 .

[55]  Roman Kuc,et al.  Introduction to Digital Signal Processing , 1988 .

[56]  John G. Proakis,et al.  Introduction to Digital Signal Processing , 1988 .

[57]  I. Pinelis,et al.  Remarks on Inequalities for Large Deviation Probabilities , 1986 .

[58]  C. L. Mallows.  Some comments on C_p , 1973 .

[59]  Walter D. Fisher.  On Grouping for Maximum Homogeneity , 1958 .