Regularized Kernel Algorithms for Support Estimation

In the framework of non-parametric support estimation, we study the statistical properties of an estimator defined by means of Kernel Principal Component Analysis (KPCA). In the context of anomaly/novelty detection the algorithm was first introduced by Hoffmann in 2007. We also extend to above analysis to a larger class of set estimators defined in terms of a filter function.

[1]  L. Devroye,et al.  Detection of Abnormal Behavior Via Nonparametric Estimation of the Support , 1980 .

[2]  A. Tsybakov,et al.  Minimax theory of image reconstruction , 1993 .

[3]  G. Folland.  A course in abstract harmonic analysis , 1995 .

[4]  L. Dümbgen,et al.  RATES OF CONVERGENCE FOR RANDOM APPROXIMATIONS OF CONVEX SETS , 1996 .

[5]  A. Tsybakov.  On nonparametric estimation of density level sets , 1997 .

[6]  A. Cuevas,et al.  A plug-in approach to support estimation , 1997 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  Bernhard Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, International Conference on Machine Learning.

[9]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[10]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[11]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[12]  Matthias Reitzner,et al.  Random polytopes and the Efron--Stein jackknife inequality , 2003 .

[13]  Michael Solomyak,et al.  Double Operator Integrals in a Hilbert Space , 2003 .

[14]  Antonio Cuevas,et al.  Set estimation: an overview and some recent developments , 2003 .

[15]  Gilles Blanchard,et al.  Statistical properties of Kernel Prinicipal Component Analysis , 2019 .

[16]  Don R. Hush,et al.  A Classification Framework for Anomaly Detection , 2005, J. Mach. Learn. Res..

[17]  Robert D. Nowak,et al.  Learning Minimum Volume Sets , 2005, J. Mach. Learn. Res..

[18]  Gilles Blanchard,et al.  On the Convergence of Eigenspaces in Kernel Principal Component Analysis , 2005, NIPS.

[19]  Lorenzo Rosasco,et al.  Learning from Examples as an Inverse Problem , 2005, J. Mach. Learn. Res..

[20]  Jean-Philippe Vert,et al.  Consistency and Convergence Rates of One-Class SVMs and Related Algorithms , 2006, J. Mach. Learn. Res..

[21]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[22]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[23]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[24]  Mark R. Morelande,et al.  Statistical analysis of motion patterns in AIS Data: Anomaly detection and motion prediction , 2008, 2008 11th International Conference on Information Fusion.

[25]  Hyoungjoo Lee,et al.  Supporting diagnosis of attention-deficit hyperactive disorder with novelty detection , 2008, Artif. Intell. Medicine.

[26]  Lorenzo Rosasco,et al.  Spectral Algorithms for Supervised Learning , 2008, Neural Computation.

[27]  Wanqing Li,et al.  Kernel PCA of HOG features for posture detection , 2009, 2009 24th International Conference Image and Vision Computing New Zealand.

[28]  Mauricio Maestri,et al.  Kernel PCA Performance in Processes with Multiple Operation Modes , 2009 .

[29]  Fei He,et al.  Research on Nonlinear Process Monitoring and Fault Diagnosis Based on Kernel Principal Component Analysis , 2009 .

[30]  Bruno Pelletier,et al.  Asymptotic Normality in Density Support Estimation , 2009 .

[31]  M. U. Kurse,et al.  Computational Models for Neuromuscular Function , 2009, IEEE Reviews in Biomedical Engineering.

[32]  W. Kendall,et al.  New Perspectives in Stochastic Geometry , 2010 .

[33]  J. Andrew Bagnell,et al.  Anytime online novelty detection for vehicle safeguarding , 2010, 2010 IEEE International Conference on Robotics and Automation.

[34]  Lorenzo Rosasco,et al.  Spectral Regularization for Support Estimation , 2010, NIPS.

[35]  Ernesto De Vito,et al.  A consistent algorithm to solve Lasso, elastic-net and Tikhonov regularization , 2011, J. Complex..

[36]  E. D. Vito,et al.  Learning Sets with Separating Kernels , 2012, 1204.3573.

[37]  Lorenzo Rosasco,et al.  On the Sample Complexity of Subspace Learning , 2013, NIPS.

[38]  Johan A. K. Suykens,et al.  Regularization, Optimization, Kernels, and Support Vector Machines , 2014 .

[39]  Francesca Odone,et al.  Geometrical and computational aspects of Spectral Support Estimation for novelty detection , 2014, Pattern Recognit. Lett..

[40]  S. Mukherjee,et al.  Topological Consistency via Kernel Estimation , 2014, 1407.5272.

[41]  Arthur Zimek,et al.  On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study , 2016, Data Mining and Knowledge Discovery.

[42]  Lorenzo Rosasco,et al.  Less is More: Nyström Computational Regularization , 2015, NIPS.

[43]  Lorenzo Rosasco,et al.  NYTRO: When Subsampling Meets Early Stopping , 2015, AISTATS.

[44]  Lorenzo Rosasco,et al.  FALKON: An Optimal Large Scale Kernel Method , 2017, NIPS.

[45]  Lorenzo Rosasco,et al.  Generalization Properties of Learning with Random Features , 2016, NIPS.

[46]  Gilles Blanchard,et al.  Optimal Rates for Regularization of Statistical Inverse Learning Problems , 2016, Found. Comput. Math..