The game theoretic p-Laplacian and semi-supervised learning with few labels

We study the game theoretic p-Laplacian for semi-supervised learning on graphs, and show that it is well-posed in the limit of finite labeled data and infinite unlabeled data. In particular, we show that the continuum limit of graph-based semi-supervised learning with the game theoretic p-Laplacian is a weighted version of the continuous p-Laplace equation. We also prove that solutions to the graph p-Laplace equation are approximately Holder continuous with high probability. Our proof uses the viscosity solution machinery and the maximum principle on a graph.

[1]  Adam M. Oberman,et al.  Nonlinear elliptic Partial Differential Equations and p-harmonic functions on graphs , 2012, 1212.0834.

[2]  Dejan Slepcev,et al.  Analysis of $p$-Laplacian Regularization in Semi-Supervised Learning , 2017, SIAM J. Math. Anal..

[3]  Y. Peres,et al.  Tug-of-war and the infinity Laplacian , 2006, math/0605002.

[4]  Vesa Julin,et al.  A New Proof for the Equivalence of Weak and Viscosity Solutions for the p-Laplace Equation , 2011, 1104.2197.

[5]  Ulrike von Luxburg,et al.  Distance-Based Classification with Lipschitz Functions , 2004, J. Mach. Learn. Res..

[6]  Matthias Hein,et al.  Intrinsic dimensionality estimation of submanifolds in Rd , 2005, ICML.

[7]  Alfred O. Hero,et al.  A PDE-based Approach to Nondominated Sorting , 2013, SIAM J. Numer. Anal..

[8]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[9]  Jakub W. Pachocki,et al.  Solving SDD linear systems in nearly mlog1/2n time , 2014, STOC.

[10]  M. Crandall,et al.  A TOUR OF THE THEORY OF ABSOLUTELY MINIMIZING FUNCTIONS , 2004 .

[11]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[12]  Daniel A. Spielman,et al.  Algorithms for Lipschitz Learning on Graphs , 2015, COLT.

[13]  B. Nadler,et al.  Semi-supervised learning with the graph Laplacian: the limit of infinite unlabelled data , 2009, NIPS 2009.

[14]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1990, 29th IEEE Conference on Decision and Control.

[15]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[16]  Adam M. Oberman.  Finite difference methods for the Infinity Laplace and p-Laplace equations , 2011, 1107.5278.

[17]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[18]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[19]  Alfred O. Hero,et al.  Determining Intrinsic Dimension and Entropy of High-Dimensional Shape Spaces , 2006, Statistics and Analysis of Shapes.

[20]  Xiaojin Zhu,et al.  p-voltages: Laplacian Regularization for Semi-Supervised Learning on High-Dimensional Data , 2013 .

[21]  Bernhard Schölkopf,et al.  Learning from labeled and unlabeled data on a directed graph , 2005, ICML.

[22]  Mikhail Belkin,et al.  An iterated graph laplacian approach for ranking on manifolds , 2011, KDD.

[23]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[24]  Yang Wang,et al.  Multi-Manifold Ranking: Using Multiple Features for Better Image Retrieval , 2013, PAKDD.

[25]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[26]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[27]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[28]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[29]  A. Singer.  From graph to manifold Laplacian: The convergence rate , 2006 .

[30]  Chun Chen,et al.  Efficient manifold ranking for image retrieval , 2011, SIGIR.

[31]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[33]  P. Lindqvist.  Notes on the p-Laplace equation , 2006 .

[34]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[35]  Ulrike von Luxburg,et al.  Phase transition in the family of p-resistances , 2011, NIPS.

[36]  Jeff Calder,et al.  Consistency of Lipschitz learning with infinite unlabeled data and finite labeled data , 2017, SIAM J. Math. Data Sci..

[37]  Hitoshi Ishii,et al.  A class of integral equations and approximation of p-Laplace equations , 2010 .

[38]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[39]  Ling Huang,et al.  An Analysis of the Convergence of Graph Laplacians , 2010, ICML.

[40]  Jingrui He,et al.  Generalized Manifold-Ranking-Based Image Retrieval , 2006, IEEE Transactions on Image Processing.

[41]  Tong Zhang,et al.  Learning on Graph with Laplacian Regularization , 2006, NIPS.

[42]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[43]  Mikhail Belkin,et al.  Towards a Theoretical Foundation for Laplacian-Based Manifold Methods , 2005, COLT.

[44]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[45]  Jingrui He,et al.  Manifold-ranking based image retrieval , 2004, MULTIMEDIA '04.

[46]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[47]  Petri Juutinen,et al.  On the Equivalence of Viscosity Solutions and Weak Solutions for a Quasi-Linear Equation , 2001, SIAM J. Math. Anal..

[48]  Matthias Hein,et al.  Uniform Convergence of Adaptive Graph-Based Regularization , 2006, COLT.

[49]  Gary M. Lieberman,et al.  Boundary regularity for solutions of degenerate elliptic equations , 1988 .

[50]  Marta Lewicka,et al.  Game theoretical methods in PDEs , 2014 .

[51]  P. Tolksdorf,et al.  Regularity for a more general class of quasilinear elliptic equations , 1984 .

[52]  J Manfredi Juan,et al.  Nonlinear elliptic Partial Differential Equations and p-harmonic functions on graphs , 2015 .

[53]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.