Local Adaptive Importance Sampling for Multivariate Densities with Strong Nonlinear Relationships

Abstract We consider adaptive importance sampling techniques that use kernel density estimates at each iteration as importance sampling functions. These can provide more nearly constant importance weights and more precise estimates of quantities of interest than the sampling importance resampling algorithm when the initial importance sampling function is diffuse relative to the target. We propose a new method that adapts to the varying local structure of the target. When the target has unusual structure, such as strong nonlinear relationships between variables, this method provides estimates with smaller mean squared error than alternative methods.

[1]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[2]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[3]  R. Lewontin,et al.  Estimation of the Number of Different Classes in a Population , 1956 .

[4]  J. Hammersley,et al.  Monte Carlo Methods , 1965 .

[5]  Philip Rabinowitz,et al.  Methods of Numerical Integration , 1985 .

[6]  V. A. Epanechnikov.  Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[7]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[8]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[9]  T. Kloek,et al.  Bayesian estimates of equation system parameters, An application of integration by Monte Carlo , 1976 .

[10]  P. Billingsley,et al.  Probability and Measure , 1980 .

[11]  Leland Stewart,et al.  Multiparameter Univariate Bayesian Analysis , 1979 .

[12]  H. K. Van Dijk,et al.  Further experience in Bayesian analysis using Monte Carlo integration , 1980 .

[13]  Leland Stewart,et al.  Bayesian Analysis Using Monte Carlo Integration - a Powerful Methodology for Handling Some Difficult Problems , 1983 .

[14]  James O. Berger,et al.  Statistical Decision Theory and Bayesian Analysis, Second Edition , 1985 .

[15]  L. Devroye,et al.  Nonparametric Density Estimation: The L 1 View. , 1985 .

[16]  Patrick Billingsley,et al.  Probability and Measure. , 1986 .

[17]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[18]  Donald B. Rubin,et al.  Comment : A noniterative sampling/importance resampling alternative to the data augmentation algorithm for creating a few imputations when fractions of missing information are modest : The SIR Algorithm , 1987 .

[19]  A. F. M. Smith,et al.  Progress with numerical and graphical methods for practical Bayesian statistics , 1987 .

[20]  L. Devroye.  A Course in Density Estimation , 1987 .

[21]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[22]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[23]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[24]  J. Geweke,et al.  Antithetic acceleration of Monte Carlo integration in Bayesian inference , 1988 .

[25]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[26]  D. Rubin.  Using the SIR algorithm to simulate posterior distributions , 1988 .

[27]  ON THE STRONG CONSISTENCY OF A NEW KERNEL PROBABILITY DENSITY ESTIMATOR , 1989 .

[28]  J. Shao.  Monte Carlo Approximations in Bayesian Decision Theory , 1989 .

[29]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[30]  M. C. Jones.  Discretized and Interpolated Kernel Density Estimates , 1989 .

[31]  G. Terrell.  The Maximal Smoothing Principle in Density Estimation , 1990 .

[32]  A. Izenman.  Review Papers: Recent Developments in Nonparametric Density Estimation , 1991 .

[33]  A. Izenman.  Recent Developments in Nonparametric Density Estimation , 1991 .

[34]  Tim Hesterberg,et al.  Importance sampling for Bayesian estimation , 1992 .

[35]  Man-Suk Oh,et al.  Adaptive importance sampling in monte carlo integration , 1992 .

[36]  W. Gilks,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 1992 .

[37]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[38]  Alan E. Gelfand,et al.  Bayesian statistics without tears: A sampling-resampling perspective , 1992 .

[39]  M. West.  Approximating posterior distributions by mixtures , 1993 .

[40]  A. Raftery,et al.  A bayesian framework and importance sampling methods for synthesizing multiple sources of evidence and uncertainty linked by a complex mechanistic model , 1993 .

[41]  Adrian E. Raftery,et al.  Inference from a Deterministic Population Dynamics Model for Bowhead Whales , 1995 .

[42]  Geof H. Givens,et al.  Consistency of the local kernel density estimator , 1995 .

[43]  Ping Zhang.  Nonparametric Importance Sampling , 1996 .