Multiclass Semi-Supervised Learning on Graphs using Ginzburg-Landau Functional Minimization

We present a graph-based variational algorithm for classification of high-dimensional data, generalizing the binary diffuse interface model to the case of multiple classes. Motivated by total variation techniques, the method involves minimizing an energy functional made up of three terms. The first two terms promote a stepwise continuous classification function with sharp transitions between classes, while preserving symmetry among the class labels. The third term is a data fidelity term, allowing us to incorporate prior information into the model in a semi-supervised framework. The performance of the algorithm on synthetic data, as well as on the COIL and MNIST benchmark datasets, is competitive with state-of-the-art graph-based multiclass segmentation methods.

[1]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[2]  Matthias Hein,et al.  Spectral clustering based on the graph p-Laplacian , 2009, ICML '09.

[3]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[4]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[5]  Arjuna Flenner,et al.  Diffuse Interface Models on Graphs for Classification of High Dimensional Data , 2012, Multiscale Model. Simul..

[6]  Andrea L. Bertozzi,et al.  Wavelet analogue of the Ginzburg–Landau energy and its Γ-convergence , 2010 .

[7]  Jeff A. Bilmes,et al.  Semi-Supervised Learning with Measure Propagation , 2011, J. Mach. Learn. Res..

[8]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[9]  Arthur D. Szlam,et al.  Total variation and cheeger cuts , 2010, ICML 2010.

[10]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[11]  Robert V. Kohn,et al.  Local minimisers and singular perturbations , 1989, Proceedings of the Royal Society of Edinburgh: Section A Mathematics.

[12]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Andrea L. Bertozzi,et al.  Inpainting of Binary Images Using the Cahn–Hilliard Equation , 2007, IEEE Transactions on Image Processing.

[14]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[15]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[16]  Yoon Mo Jung,et al.  Multiphase Image Segmentation via Modica-Mortola Phase Transition , 2007, SIAM J. Appl. Math..

[17]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[18]  Dan Roth,et al.  Constraint Classification for Multiclass Classification and Ranking , 2002, NIPS.

[19]  Ronald R. Coifman,et al.  Regularization on Graphs with Function-adapted Diffusion Processes , 2008, J. Mach. Learn. Res..

[20]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[21]  Andrea L. Bertozzi,et al.  A Wavelet-Laplace Variational Technique for Image Deconvolution and Inpainting , 2008, IEEE Transactions on Image Processing.

[22]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[23]  A. Bertozzi,et al.  $\Gamma$-convergence of graph Ginzburg-Landau functionals , 2012, Advances in Differential Equations.

[24]  Matthias Hein,et al.  Beyond Spectral Clustering - Tight Relaxations of Balanced Graph Cuts , 2011, NIPS.

[25]  Shih-Fu Chang,et al.  Graph transduction via alternating minimization , 2008, ICML '08.

[26]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[27]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[28]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[29]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[30]  Yibao Li,et al.  Multiphase image segmentation using a phase-field model , 2011, Comput. Math. Appl..