Continuum Limits of Posteriors in Graph Bayesian Inverse Problems

We consider the problem of recovering a function input of a differential equation formulated on an unknown domain $M$. We assume to have access to a discrete domain $M_n=\{x_1, \dots, x_n\} \subset M$, and to noisy measurements of the output solution at $p\le n$ of those points. We introduce a graph-based Bayesian inverse problem, and show that the graph-posterior measures over functions in $M_n$ converge, in the large $n$ limit, to a posterior over functions in $M$ that solves a Bayesian inverse problem with known domain. The proofs rely on the variational formulation of the Bayesian update, and on a new topology for the study of convergence of measures over functions on point clouds to a measure over functions on the continuum. Our framework, techniques, and results may serve to lay the foundations of robust uncertainty quantification of graph-based tasks in machine learning. The ideas are presented in the concrete setting of recovering the initial condition of the heat equation on an unknown manifold.

[1]  P. Meyer Probability and potentials , 1966 .

[2]  A. Zellner Optimal Information Processing and Bayes's Theorem , 1988 .

[3]  G. D. Maso,et al.  An Introduction to-convergence , 1993 .

[4]  Y. Marzouk,et al.  Large-Scale Inverse Problems and Quantification of Uncertainty , 1994 .

[5]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[6]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[7]  James C. Robinson,et al.  Bayesian inverse problems for functions and applications to fluid mechanics , 2009 .

[8]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[9]  Nicola Gigli,et al.  On the heat flow on metric measure spaces: existence, uniqueness and stability , 2010 .

[10]  Andrew M. Stuart,et al.  Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem , 2011, SIAM J. Numer. Anal..

[11]  Sylvia Serfaty,et al.  Gamma-convergence of gradient flows on Hilbert and metric spaces and applications , 2011 .

[12]  E. Feinberg,et al.  Fatou's Lemma for Weakly Converging Probabilities , 2012, 1206.4073.

[13]  Sergei Ivanov,et al.  A graph discretization of the Laplace-Beltrami operator , 2013, 1301.2222.

[14]  Nicolás García Trillos,et al.  On the rate of convergence of empirical measures in $\infty$-transportation distance , 2014, 1407.1157.

[15]  O. Papaspiliopoulos,et al.  Importance Sampling: Intrinsic Dimension and Computational Cost , 2015, 1511.06196.

[16]  Andrew M. Stuart,et al.  Importance Sampling: Computational Complexity and Intrinsic Dimension , 2015 .

[17]  Nicolás García Trillos,et al.  Continuum Limit of Total Variation on Point Clouds , 2014, Archive for Rational Mechanics and Analysis.

[18]  Nicolas Garcia Trillos,et al.  The Bayesian formulation and well-posedness of fractional elliptic inverse problems , 2016, 1611.05475.

[19]  Andrew M. Stuart,et al.  Uncertainty Quantification in the Classification of High Dimensional Data , 2017, ArXiv.

[20]  Nicolas Garcia Trillos,et al.  Gradient Flows: Applications to Classification, Image Denoising, and Riemannian MCMC , 2017 .

[21]  Dejan Slepcev,et al.  A variational approach to the consistency of spectral clustering , 2015, Applied and Computational Harmonic Analysis.