Classification via regularization on graphs

We present a novel data classifier that is based on the regularization of graph signals. Our approach is based on the theory of discrete signal processing on graphs where the graph represents similarities between data and we interpret labels for the dataset elements as a signal indexed by the nodes of the graph. We postulate that true labels form a low-frequency graph signal and the classifier finds the smoothest graph signal that satisfies constraints given by known data labels. Our experiments demonstrate that our approach achieves high accuracy in multiclass classification and outperforms other classification approaches.

[1]  José M. F. Moura,et al.  The Algebraic Approach to the Discrete Cosine and Sine Transforms and Their Fast Algorithms , 2003, SIAM J. Comput..

[2]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[3]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[4]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[5]  Yue M. Lu,et al.  A Spectral Graph Uncertainty Principle , 2012, IEEE Transactions on Information Theory.

[6]  Markus Püschel,et al.  Algebraic Signal Processing Theory: Foundation and 1-D Time , 2008, IEEE Transactions on Signal Processing.

[7]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[8]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[9]  Tony F. Chan,et al.  The digital TV filter and nonlinear denoising , 2001, IEEE Trans. Image Process..

[10]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[11]  José M. F. Moura,et al.  Algebraic Signal Processing Theory: 1-D Space , 2008, IEEE Transactions on Signal Processing.

[12]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[13]  Michael G. Rabbat,et al.  Approximating signals supported on graphs , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Mikhail Belkin,et al.  Tikhonov regularization and semi-supervised learning on large graphs , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[15]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[16]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[17]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[18]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[19]  José M. F. Moura,et al.  Discrete signal processing on graphs: Graph filters , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Jelena Kovacevic,et al.  Algebraic Signal Processing Theory: 1-D Nearest Neighbor Models , 2012, IEEE Transactions on Signal Processing.

[21]  José M. F. Moura,et al.  Discrete signal processing on graphs: Graph fourier transform , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.