Spectral Embedding Norm: Looking Deep into the Spectrum of the Graph Laplacian

The extraction of clusters from a dataset which includes multiple clusters and a significant background component is a non-trivial task of practical importance. In image analysis this manifests for example in anomaly detection and target detection. The traditional spectral clustering algorithm, which relies on the leading $K$ eigenvectors to detect $K$ clusters, fails in such cases. In this paper we propose the {\it spectral embedding norm} which sums the squared values of the first $I$ normalized eigenvectors, where $I$ can be significantly larger than $K$. We prove that this quantity can be used to separate clusters from the background in unbalanced settings, including extreme cases such as outlier detection. The performance of the algorithm is not sensitive to the choice of $I$, and we demonstrate its application on synthetic and real-world remote sensing and neuroimaging datasets.

[1]  Amir Averbuch,et al.  PCA-Based Out-of-Sample Extension for Dimensionality Reduction , 2015, 1511.00831.

[2]  Maoguo Gong,et al.  Spectral clustering with eigenvector selection based on entropy ranking , 2010, Neurocomputing.

[3]  Tosio Kato Perturbation theory for linear operators , 1966 .

[4]  Bin Yu,et al.  The geometry of kernelized spectral clustering , 2014, 1404.7552.

[5]  Ronald R. Coifman,et al.  Automated cellular structure extraction in biological images with applications to calcium imaging data , 2018, bioRxiv.

[6]  Matthias Bethge,et al.  Standardizing and benchmarking data analysis for calcium imaging , 2017 .

[7]  Israel Cohen,et al.  Iterative diffusion-based anomaly detection , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Shaogang Gong,et al.  Spectral clustering with eigenvector selection , 2008, Pattern Recognit..

[9]  Stefan Steinerberger,et al.  The Geometry of Nodal Sets and Outlier Detection , 2017, 1706.01362.

[10]  T. Coleman,et al.  Auto insurance fraud detection using unsupervised spectral ranking for anomaly , 2016 .

[11]  Yoel Shkolnisky,et al.  Diffusion Interpretation of Nonlocal Neighborhood Filters for Signal Denoising , 2009, SIAM J. Imaging Sci..

[12]  Michael W. Mahoney,et al.  Localization on low-order eigenvectors of data matrices , 2011, ArXiv.

[13]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[14]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Quico Spaen,et al.  HNCcorr: A Novel Combinatorial Approach for Cell Identification in Calcium-Imaging Movies , 2017, eNeuro.

[16]  Jean-Michel Morel,et al.  Image Anomalies: A Review and Synthesis of Detection Methods , 2019, Journal of Mathematical Imaging and Vision.

[17]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[18]  Alexander Cloninger,et al.  Diffusion Nets , 2015, Applied and Computational Harmonic Analysis.

[19]  Spencer L. Smith,et al.  Parallel processing of visual space by neighboring neurons in mouse visual cortex , 2010, Nature Neuroscience.

[20]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[21]  Naoki Saito,et al.  How Can We Naturally Order and Organize Graph Laplacian Eigenvectors? , 2018, 2018 IEEE Statistical Signal Processing Workshop (SSP).

[22]  Carmeline J. Dsilva,et al.  Parsimonious Representation of Nonlinear Dynamical Systems Through Manifold Learning: A Chemotaxis Case Study , 2015, 1505.06118.

[23]  Israel Cohen,et al.  Multiscale Anomaly Detection Using Diffusion Maps , 2013, IEEE Journal of Selected Topics in Signal Processing.

[24]  Lexing Ying,et al.  Robust and efficient multi-way spectral clustering , 2016, ArXiv.

[25]  Seiichi Uchida,et al.  A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data , 2016, PloS one.

[26]  Amir Averbuch,et al.  Detection of Anomaly Trends in Dynamically Evolving Systems , 2010, AAAI Fall Symposium: Manifold Learning and Its Applications.

[27]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[28]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[29]  Meirav Galun,et al.  Fundamental Limitations of Spectral Clustering , 2006, NIPS.

[30]  R. Welsch,et al.  The Hat Matrix in Regression and ANOVA , 1978 .

[31]  Wojciech Czaja,et al.  Eigenvector localization on data-dependent graphs , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[32]  Zhi-Hua Zhou,et al.  A spectral approach to detecting subtle anomalies in graphs , 2013, Journal of Intelligent Information Systems.

[33]  Pier Luigi Dragotti,et al.  ABLE: An Activity-Based Level Set Segmentation Algorithm for Two-Photon Calcium Imaging Data , 2017, eNeuro.

[34]  Lin Tian,et al.  Neural activity imaging with genetically encoded calcium indicators. , 2012, Progress in brain research.

[35]  David Zhang,et al.  A survey of graph theoretical approaches to image segmentation , 2013, Pattern Recognit..

[36]  Hisashi Kashima,et al.  Eigenspace-based anomaly detection in computer systems , 2004, KDD.

[37]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[38]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[39]  B. Nadler,et al.  Diffusion Maps - a Probabilistic Interpretation for Spectral Embedding and Clustering Algorithms , 2008 .

[40]  Adam M. Packer,et al.  Extracting regions of interest from biological images with convolutional sparse block coding , 2013, NIPS.

[41]  Patrick J. Wolfe,et al.  Subgraph Detection Using Eigenvector L1 Norms , 2010, NIPS.

[42]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[43]  C. Oliveira,et al.  Eigenfunction entropy as a criterion of localization , 2002 .

[44]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[45]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..