Bayesian latent variable regression via Gibbs sampling: methodology and practical aspects

Bayesian latent variable regression (BLVR) aims to utilize all available information for empirical modeling via a Bayesian framework. Such information includes prior knowledge about the underlying variables, model parameters and measurement error distributions. This paper improves upon the existing optimization‐based BLVR (BLVR‐OPT) method [1] by developing a sampling‐based Bayesian latent variable regression (BLVR‐S) method that relies on Gibbs sampling. Use of the sampling‐based framework not only provides point estimates, but its ability to generate samples that represent the posterior distribution of the unknown variables, also readily provides error bounds. Features and advantages of this method are demonstrated via examples based on simulated data and real Near‐Infrared (NIR) spectroscopy data. Practical aspects of Bayesian modeling such as determining when the extra computation may be worth the effort are addressed by an empirical study of the effects of the amount of training data and signal to noise ratio (SNR). The benefits of BLVR seem to be most significant when the number of measurements is limited and when noise in output variables is relatively large. Copyright © 2007 John Wiley & Sons, Ltd.

[1]  Hsin-I Lin,et al.  A bayesian approach to parameter estimation for a crayfish (Procambarus spp): Bioaccumulation model , 2004, Environmental toxicology and chemistry.

[2]  Prem K. Goel,et al.  Process modeling by Bayesian latent variable regression , 2002 .

[3]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[4]  Thomas A. Duever,et al.  Parameter Estimation in the Error‐in‐Variables Models Using the Gibbs Sampler , 2008 .

[5]  Darren T. Andrews,et al.  Maximum likelihood principal component analysis , 1997 .

[6]  Peter Guttorp,et al.  Multivariate receptor models and model uncertainty , 2002 .

[7]  J. Shao,et al.  The jackknife and bootstrap , 1996 .

[8]  Paul H. Garthwaite,et al.  Quantifying and using expert opinion for variable-selection problems in regression , 1996 .

[9]  Matthew C. Coleman,et al.  Bayesian parameter estimation with informative priors for nonlinear systems , 2006 .

[10]  Alexey L. Pomerantsev,et al.  Prediction of the aging of polymer materials , 1999 .

[11]  Beata Walczak,et al.  Comparison of Multivariate Calibration Techniques Applied to Experimental NIR Data Sets , 2000 .

[12]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[13]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[14]  Jeff Gill,et al.  Elicited Priors for Bayesian Model Specifications in Political Science Research , 2005 .

[15]  J. Edward Jackson,et al.  A User's Guide to Principal Components: Jackson/User's Guide to Principal Components , 2004 .

[16]  Alison J. Burnham,et al.  LATENT VARIABLE MULTIVARIATE REGRESSION MODELING , 1999 .

[17]  J. Kalivas.  Two data sets of near infrared spectra , 1997 .

[18]  Svante Wold,et al.  Partial least-squares method for spectrofluorimetric analysis of mixtures of humic acid and lignin sulfonate , 1983 .

[19]  Alexey L. Pomerantsev,et al.  Non‐linear regression analysis: new approach to traditional implementations , 2000 .

[20]  B. Bakshi,et al.  Bayesian estimation via sequential Monte Carlo sampling: Unconstrained nonlinear dynamic systems , 2004 .

[21]  B. Bakshi,et al.  Bayesian principal component analysis , 2002 .

[22]  J. E. Jackson.  A User's Guide to Principal Components , 1991 .

[23]  Darren T. Andrews,et al.  Maximum Likelihood Multivariate Calibration , 2022 .

[24]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[25]  Peter A. Vanrolleghem,et al.  Determining environmental standards using bootstrapping, bayesian and maximum likelihood techniques : a comparative study , 2001 .

[26]  D. Coomans,et al.  Recent developments in discriminant analysis on high dimensional spectral data , 1996 .

[27]  L. Mark Berliner,et al.  Subsampling the Gibbs Sampler , 1994 .

[28]  Mohamed Nounou.  Dealing with Collinearity in Finite Impulse Response Models Using Bayesian Shrinkage , 2006 .

[29]  Mordechai Jaeger,et al.  An application of a Bayesian approach to the combination of measurements of different accuracies , 1997 .