Adaptive rejection Metropolis sampling using Lagrange interpolation polynomials of degree 2

A crucial problem in Bayesian posterior computation is efficient sampling from a univariate distribution, e.g. a full conditional distribution in applications of the Gibbs sampler. This full conditional distribution is usually non-conjugate, algebraically complex and computationally expensive to evaluate. We propose an alternative algorithm, called ARMS2, to the widely used adaptive rejection sampling technique ARS [Gilks, W.R., Wild, P., 1992. Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41 (2), 337-348; Gilks, W.R., 1992. Derivative-free adaptive rejection sampling for Gibbs sampling. In: Bernardo, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (Eds.), Bayesian Statistics, Vol. 4. Clarendon, Oxford, pp. 641-649] for generating a sample from univariate log-concave densities. Whereas ARS is based on sampling from piecewise exponentials, the new algorithm uses truncated normal distributions and makes use of a clever auxiliary variable technique [Damien, P., Walker, S.G., 2001. Sampling truncated normal, beta, and gamma densities. Journal of Computational and Graphical Statistics 10 (2) 206-215]. Furthermore, we extend this algorithm to deal with non-log-concave densities to provide an enhanced alternative to adaptive rejection Metropolis sampling, ARMS [Gilks, W.R., Best, N.G., Tan, K.K.C., 1995. Adaptive rejection Metropolis sampling within Gibbs sampling. Applied Statistics 44, 455-472]. The performance of ARMS and ARMS2 is compared in simulations of standard univariate distributions as well as in Gibbs sampling of a Bayesian hierarchical state-space model used for fisheries stock assessment.

[1]  W. Gilks,et al.  Adaptive Rejection Metropolis Sampling Within Gibbs Sampling , 1995 .

[2]  E. K. Gatcombe.  Discussion: “The Measurement of Oil-Film Thickness in Gear Teeth” (MacConochie, I. O., and Cameron, A., 1960, ASME J. Basic Eng., 82, pp. 29–34) , 1960 .

[3]  Adrian F. M. Smith,et al.  Efficient generation of random variates via the ratio-of-uniforms method , 1991 .

[4]  Meyer,et al.  Bayesian reconstruction of chaotic dynamical systems , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[5]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[6]  M. West,et al.  Bayesian forecasting and dynamic models , 1989 .

[7]  George Marsaglia.  Generating discrete random variables in a computer , 1963, CACM.

[8]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[9]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[10]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[11]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[12]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[13]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[14]  Jun Yu,et al.  Bugs for a Bayesian Analysis of Stochastic Volatility Models , 2000 .

[15]  Adrian F. M. Smith,et al.  Bayesian Inference for Generalized Linear and Proportional Hazards Models Via Gibbs Sampling , 1993 .

[16]  Hans Kiinsch,et al.  State Space and Hidden Markov Models , 2000 .

[17]  Russell B. Millar,et al.  Bayesian stock assessment using a state-space implementation of the delay difference model , 1999 .

[18]  D. Cox,et al.  Complex stochastic systems , 2000 .

[19]  C. Walters,et al.  Quantitative fisheries stock assessment: Choice, dynamics and uncertainty , 2004, Reviews in Fish Biology and Fisheries.

[20]  B. Schmeiser,et al.  Toward Black-Box Sampling: A Random-Direction Interior-Point Markov Chain Approach , 1998 .

[21]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[22]  J. Pella,et al.  A generalized stock production model , 1969 .

[23]  W. Gilks,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 1992 .

[24]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[25]  Adrian F. M. Smith,et al.  Bayesian Analysis of Linear and Non‐Linear Population Models by Using the Gibbs Sampler , 1994 .

[26]  D. Estep.  Practical Analysis in One Variable , 2002 .

[27]  S. Walker.  Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[28]  L. Fahrmeir,et al.  Multivariate statistical modelling based on generalized linear models , 1994 .

[29]  P. Dellaportas.  Random variate transformations in the Gibbs sampler: issues of efficiency and convergence , 1995 .

[30]  S. Walker,et al.  Sampling Truncated Normal, Beta, and Gamma Densities , 2001 .

[31]  L. Tierney,et al.  Efficiency and Convergence Properties of Slice Samplers , 2002 .

[32]  Nicholas G. Polson,et al.  A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling , 1992 .

[33]  田中 豊.  Multivariate Statistical Modelling Based on Generalized Linear Models/Ludwig Fahrmeir,Gerhard Tutz(1994) , 1995 .

[34]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  J. E. Norman,et al.  A computer program for the generation of random variables from any discrete distribution , 1972 .