An overview on the Gaussian Fields and Harmonic Functions method for semi-supervised learning

Graph-based semi-supervised learning (SSL) algorithms have gained increased attention in the last few years due to their high classification performance on many application domains. One of the widely used methods for graph-based SSL is the Gaussian Fields and Harmonic Functions (GFHF), which is formulated as an optimization problem using a Laplacian regularizer term with a fitting constraint on labeled examples. Such a method and its variations were effectively applied on many fields of machine learning, such as active learning and dimensionality reduction. In this paper, we provide an overview on the GFHF algorithm, focusing on its regularization framework, convergence analysis, out-of-sample extension, scalability, and active learning. We also provide an experimental analysis on inductive SSL in order to show that we can effectively classify out-of-sample examples using the GFHF algorithm without the necessity of using kernel expansions.

[1]  Mikhail Belkin,et al.  Semi-supervised Learning by Higher Order Regularization , 2011, AISTATS.

[2]  Matthias Hein,et al.  Manifold Denoising , 2006, NIPS.

[3]  J. Lafferty,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[4]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[5]  Zhi-Hua Zhou,et al.  A New Analysis of Co-Training , 2010, ICML.

[6]  Wei Liu,et al.  Robust multi-class transductive learning with graphs , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Fei Wang,et al.  Efficient label propagation for interactive image segmentation , 2007, Sixth International Conference on Machine Learning and Applications (ICMLA 2007).

[8]  Celso André Rodrigues de Sousa,et al.  Analysis of the backpropagation algorithm using linear algebra , 2012, IJCNN.

[9]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[10]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[11]  Dongxiao Zhu,et al.  Improvement of Bayesian Network Inference Using a Relaxed Gene Ordering , 2007, ICMLA 2007.

[12]  Ameet Talwalkar,et al.  Ensemble Nystrom Method , 2009, NIPS.

[13]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[14]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[15]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[16]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[17]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[18]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[19]  Jiawei Han,et al.  A Variance Minimization Criterion to Active Learning on Graphs , 2012, AISTATS.

[20]  Gustavo Camps-Valls,et al.  Semi-Supervised Graph-Based Hyperspectral Image Classification , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[22]  Gustavo E. A. P. A. Batista,et al.  Robust Multi-class Graph Transduction with higher order regularization , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[23]  James T. Kwok,et al.  Making Large-Scale Nyström Approximation Possible , 2010, ICML.

[24]  James T. Kwok,et al.  Prototype vector machine for large scale semi-supervised learning , 2009, ICML '09.

[25]  Feiping Nie,et al.  Semi-supervised Dimensionality Reduction via Harmonic Functions , 2011, MDAI.

[26]  Alexandre Cardoso,et al.  Development of Adaptive Information Visualization Systems with Augmented Reality , 2014, 2014 18th International Conference on Information Visualisation.

[27]  Celso André R. de Sousa,et al.  An experimental analysis on time series transductive classification on graphs , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[28]  Celso André R. de Sousa,et al.  Influence of Graph Construction on Semi-supervised Learning , 2013, ECML/PKDD.

[29]  Celso André R. de Sousa,et al.  Time Series Transductive Classification on Imbalanced Data Sets: An Experimental Study , 2014, 2014 22nd International Conference on Pattern Recognition.

[30]  Xiaojin Zhu,et al.  Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning , 2005, ICML.

[31]  Shih-Fu Chang,et al.  Graph construction and b-matching for semi-supervised learning , 2009, ICML '09.

[32]  Ivor W. Tsang,et al.  Improved Nyström low-rank approximation and error analysis , 2008, ICML '08.

[33]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[34]  Wei Liu,et al.  Large Graph Construction for Scalable Semi-Supervised Learning , 2010, ICML.

[35]  Ronald Rosenfeld,et al.  Semi-supervised learning with graphs , 2005 .

[36]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[37]  Nando de Freitas,et al.  Fast Computational Methods for Visually Guided Robots , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.