Concave-Convex Adaptive Rejection Sampling

We describe a method for generating independent samples from univariate density functions using adaptive rejection sampling without the log-concavity requirement. The method makes use of the fact that many functions can be expressed as a sum of concave and convex functions. Using a concave-convex decomposition, we bound the log-density by separately bounding the concave and convex parts using piecewise linear functions. The upper bound can then be used as the proposal distribution in rejection sampling. We demonstrate the applicability of the concave-convex approach on a number of standard distributions and describe an application to the efficient construction of sequential Monte Carlo proposal distributions for inference over genealogical trees. Computer code for the proposed algorithms is available online.

[1]  Arnaud Doucet,et al.  Sequential Monte Carlo Methods , 2006, Handbook of Graphical Models.

[2]  Radford M. Neal.  Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[3]  By W. R. GILKSt,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 2010 .

[4]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[5]  Yee Whye Teh,et al.  An Efficient Sequential Monte Carlo Algorithm for Coalescent Clustering , 2008, NIPS.

[6]  Renate Meyer,et al.  Adaptive rejection Metropolis sampling using Lagrange interpolation polynomials of degree 2 , 2008, Comput. Stat. Data Anal..

[7]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .

[8]  Yee Whye Teh,et al.  Bayesian Agglomerative Clustering with Coalescents , 2007, NIPS.

[9]  Yuguo Chen,et al.  Stopping‐time resampling for sequential Monte Carlo methods , 2005 .

[10]  James Stuart Tanton,et al.  Encyclopedia of Mathematics , 2005 .

[11]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[12]  S. Kou,et al.  Modeling growth stocks via birth-death processes , 2003, Advances in Applied Probability.

[13]  Hui Wang,et al.  First passage times of a jump diffusion process , 2003, Advances in Applied Probability.

[14]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[15]  Radford M. Neal.  Slice Sampling , 2000, physics/0009028.

[16]  Ernst Eberlein,et al.  Generalized Hyperbolic and Inverse Gaussian Distributions: Limiting Cases and Approximation of Processes , 2003 .

[17]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[18]  Montgomery Slatkin,et al.  A vectorized method of importance sampling with applications to models of mutation and migration. , 2002, Theoretical population biology.

[19]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[20]  M. Opper,et al.  Advanced mean field methods: theory and practice , 2001 .

[21]  David P.M. SCOI.I.NIK.  SIMULATING RANDOM VARIATES FROM MAKEHAM'S DISTRIBUTION AND FROM OTHERS W I T H EXACT OR NEARLY LOG-CONCAVE DENSITIES , 2000 .

[22]  P. Donnelly,et al.  Inference in molecular population genetics , 2000 .

[23]  R. Horst,et al.  DC Programming: Overview , 1999 .

[24]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[25]  M. Evans,et al.  Random Variable Generation Using Concavity Properties of Transformed Densities , 1998 .

[26]  S. Tavaré,et al.  The age of a mutation in a general coalescent tree , 1998 .

[27]  Ole E. Barndorff-Nielsen,et al.  Processes of normal inverse Gaussian type , 1997, Finance Stochastics.

[28]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[29]  W. Gilks,et al.  Adaptive Rejection Metropolis Sampling Within Gibbs Sampling , 1995 .

[30]  Wolfgang Hörmann,et al.  A rejection technique for sampling from T-concave distributions , 1995, TOMS.

[31]  Walter R. Gilks,et al.  Adaptive rejection metropolis sampling , 1995 .

[32]  Walter R. Gilks,et al.  BUGS - Bayesian inference Using Gibbs Sampling Version 0.50 , 1995 .

[33]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[34]  John Dagpunar,et al.  An Easily Implemented Generalised Inverse Gaussian Generator , 1989 .

[35]  A. Kennedy,et al.  Hybrid Monte Carlo , 1988 .

[36]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[37]  J. Kingman.  On the genealogy of large populations , 1982, Journal of Applied Probability.

[38]  B. Jørgensen.  Statistical Properties of the Generalized Inverse Gaussian Distribution , 1981 .

[39]  C. J-F,et al.  THE COALESCENT , 1980 .

[40]  O. Barndorff-Nielsen,et al.  Infinite divisibility of the hyperbolic and generalized inverse Gaussian distributions , 1977 .

[41]  Ian F. Blake,et al.  Level-crossing problems for random processes , 1973, IEEE Trans. Inf. Theory.

[42]  J. Durbin.  Boundary-crossing probabilities for the Brownian motion and Poisson processes and techniques for computing the power of the Kolmogorov-Smirnov test , 1971, Journal of Applied Probability.

[43]  Z. Ciesielski,et al.  First passage times and sojourn times for Brownian motion in space and the exact Hausdorff measure of the sample path , 1962 .

[44]  P. Hartman.  On functions representable as a difference of convex functions , 1959 .

[45]  D. Darling,et al.  THE FIRST PASSAGE PROBLEM FOR A CONTINUOUS MARKOFF PROCESS , 1953 .

[46]  W. M. Makeham,et al.  On the Law of Mortality and the Construction of Annuity Tables , 1860 .