Bayesian Prediction with Covariates Subject to Detection Limits

ABSTRACT. Missing values in covariates due to censoring by signal interference or lack of sensitivity in the measuring devices are common in industrial problems. We propose a full Bayesian solution to the prediction problem with an efficient Markov Chain Monte Carlo (MCMC) algorithm that updates all the censored covariate values jointly in a random scan Gibbs sampler. We show that the joint updating of missing covariate values can be at least two orders of magnitude more efficient than univariate updating. This increased efficiency is shown to be crucial for quickly learning the missing covariate values and their uncertainty in a real-time decision making context, in particular when there is substantial correlation in the posterior for the missing values. The approach is evaluated on simulated data and on data from the telecom sector. Our results show that the proposed Bayesian imputation gives substantially more accurate predictions than naïve imputation, and that the use of auxiliary variables in the imputation gives additional predictive power.

[1]  MinJae Lee,et al.  Multiple imputation for left‐censored biomarker data based on Gibbs sampling method , 2012, Statistics in medicine.

[2]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[3]  F. Dovis GNSS Interference Threats and Countermeasures , 2015 .

[4]  J. Singer,et al.  Simple linear regression with interval censored dependent and independent variables , 2018, Statistical methods in medical research.

[5]  S. Sinha,et al.  Estimation in generalized linear models under censored covariates with an application to MIREC data , 2018, Statistics in medicine.

[6]  Robert W. Coombs,et al.  Longitudinal Analysis of Quantitative Virologic Measures in Human Immunodeficiency Virus-Infected Subjects with ⩾400 CD4 Lymphocytes: Implications for Applying Measurements to Individual Patients , 1997 .

[7]  D. Harville Matrix Algebra From a Statistician's Perspective , 1998 .

[8]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[9]  Z. Botev The normal law under linear restrictions: simulation and estimation via minimax tilting , 2016, 1603.04166.

[10]  Henrik Ryden,et al.  Predicting strongest cell on secondary carrier using primary carrier data , 2018, 2018 IEEE Wireless Communications and Networking Conference Workshops (WCNCW).

[11]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[12]  Robert H. Lyles,et al.  Random regression models for human immunodeficiency virus ribonucleic acid data subject to left censoring and informative drop‐outs , 2000 .

[13]  Paul W. Bernhardt,et al.  Statistical Methods for Generalized Linear Models with Covariates Subject to Detection Limits , 2015, Statistics in biosciences.

[14]  U. Grenander,et al.  Comparing sweep strategies for stochastic relaxation , 1991 .

[15]  Yu Ryan Yue,et al.  Bayesian inference for generalized linear mixed models with predictors subject to detection limits: an approach that leverages information from auxiliary variables , 2016, Statistics in medicine.

[16]  Qingxia Chen,et al.  A Bayesian approach for generalized linear models with explanatory biomarker measurement variables subject to detection limit: an application to acute lung injury , 2012, Journal of applied statistics.

[17]  James G. Scott,et al.  Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables , 2012, 1205.0310.

[18]  Srikesh G. Arunajadai,et al.  Handling covariates subject to limits of detection in regression , 2012, Environmental and Ecological Statistics.

[19]  J. Hughes,et al.  Mixed Effects Models with Censored Data with Application to HIV RNA Levels , 1999, Biometrics.