Kernel PCA for novelty detection

Kernel principal component analysis (kernel PCA) is a non-linear extension of PCA. This study introduces and investigates the use of kernel PCA for novelty detection. Training data are mapped into an infinite-dimensional feature space. In this space, kernel PCA extracts the principal components of the data distribution. The squared distance to the corresponding principal subspace is the measure for novelty. This new method demonstrated a competitive performance on two-dimensional synthetic distributions and on two real-world data sets: handwritten digits and breast-cancer cytology.

[1]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[2]  W. Kissick.  Introduction to Medical Decision making , 1969 .

[3]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[4]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[5]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[8]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[9]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[10]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[11]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[12]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[13]  Heng-Da Cheng,et al.  Approaches for automated detection and classification of masses in mammograms , 2006, Pattern Recognit..

[14]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[15]  Takio Kurita,et al.  Robust De-noising by Kernel PCA , 2002, ICANN.

[16]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[17]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[18]  Michael Brady,et al.  Novelty detection for the identification of masses in mammograms , 1995 .

[19]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[20]  W. Press,et al.  Numerical Recipes in Fortran: The Art of Scientific Computing.@@@Numerical Recipes in C: The Art of Scientific Computing. , 1994 .

[21]  Sun-Yuan Kung,et al.  Principal Component Neural Networks: Theory and Applications , 1996 .

[22]  Abraham Wald,et al.  Statistical Decision Functions , 1951 .

[23]  Congde Lu,et al.  A robust kernel PCA algorithm , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[24]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[25]  Bernhard Schölkopf,et al.  Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces , 1998, DAGM-Symposium.

[26]  L. Tarassenko,et al.  Novelty detection in jet engines , 1999 .

[27]  David M. J. Tax,et al.  Kernel Whitening for One-Class Classification , 2002, Int. J. Pattern Recognit. Artif. Intell..

[28]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[29]  En Sup Yoon,et al.  Analysis of Novelty Detection Properties of Autoassociators , 2001 .