Black-box sampling for weakly smooth Langevin Monte Carlo using p-generalized Gaussian smoothing.

Discretization of continuous-time diffusion processes is a widely recognized method for sampling. However, the canonical Euler-Maruyama discretization of the Langevin diffusion process, also named as Langevin Monte Carlo (LMC), studied mostly in the context of smooth (gradient-Lipschitz) and strongly log-concave densities, a significant constraint for its deployment in many sciences, including computational statistics and statistical learning. In this paper, we establish several theoretical contributions to the literature on such sampling methods. Particularly, we generalize the Gaussian smoothing, approximate the gradient using p-generalized Gaussian smoothing and take advantage of it in the context of black-box sampling. We first present a non-strongly concave and weakly smooth black-box LMC algorithm, ideal for practical applicability of sampling challenges in a general setting.

[1]  G. Parisi.  Correlation functions and computer simulations (II) , 1981 .

[2]  Martin E. Dyer,et al.  A random polynomial-time algorithm for approximating the volume of convex bodies , 1991, JACM.

[3]  Radford M. Neal.  Bayesian Learning via Stochastic Dynamics , 1992, NIPS.

[4]  R. Tweedie,et al.  Exponential convergence of Langevin distributions and their discrete approximations , 1996 .

[5]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[6]  Santosh S. Vempala,et al.  Dispersion of Mass and the Complexity of Randomized Geometric Algorithms , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[7]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[8]  Faming Liang,et al.  Statistical and Computational Inverse Problems , 2006, Technometrics.

[9]  Santosh S. Vempala,et al.  The geometry of logconcave functions and sampling algorithms , 2007, Random Struct. Algorithms.

[10]  W. Richter.  Generalized spherical and simplicial coordinates , 2007 .

[11]  C. Villani.  Optimal Transport: Old and New , 2008 .

[12]  S. Glotzer,et al.  Time-course gait analysis of hemiparkinsonian rats following 6-hydroxydopamine lesion , 2004, Behavioural Brain Research.

[13]  Yves F. Atchad'e.  A Moreau-Yosida approximation scheme for a class of high-dimensional posterior distributions , 2015, 1505.07072.

[14]  Yurii Nesterov,et al.  Random Gradient-Free Minimization of Convex Functions , 2015, Foundations of Computational Mathematics.

[15]  Volkan Cevher,et al.  Mirrored Langevin Dynamics , 2018, NeurIPS.

[16]  Eric Moulines,et al.  Efficient Bayesian Computation by Proximal Markov Chain Monte Carlo: When Langevin Meets Moreau , 2016, SIAM J. Imaging Sci..

[17]  Martin J. Wainwright,et al.  Fast MCMC Sampling Algorithms on Polytopes , 2017, J. Mach. Learn. Res..

[18]  Michael I. Jordan,et al.  Sharp Convergence Rates for Langevin Dynamics in the Nonconvex Setting , 2018, ArXiv.

[19]  Arnak S. Dalalyan,et al.  User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient , 2017, Stochastic Processes and their Applications.

[20]  Preconditioned P-ULA for Joint Deconvolution-Segmentation of Ultrasound Images , 2019, 1903.08111.

[21]  Arnak S. Dalalyan,et al.  Bounding the error of discretized Langevin algorithms for non-strongly log-concave targets , 2019, J. Mach. Learn. Res..

[22]  Jean-Yves Tourneret,et al.  Preconditioned P-ULA for Joint Deconvolution-Segmentation of Ultrasound Images , 2019, IEEE Signal Processing Letters.

[23]  Alain Durmus,et al.  High-dimensional Bayesian inference via the unadjusted Langevin algorithm , 2016, Bernoulli.

[24]  Michael I. Jordan,et al.  Langevin Monte Carlo without Smoothness , 2019, AISTATS.