Bayesian Big Data Classification: A Review with Complements

This paper focuses on the specific problem of big data classification in a Bayesian setup using Markov Chain Monte Carlo methods. It discusses the challenges presented by the big data problems associated with classification and the existing methods to handle them. Next, a new method based on two-stage Metropolis-Hastings (MH) algorithm is proposed in this context. The purpose of of this algorithm is to reduce the exact likelihood computational cost in the big data context. In the first stage, a new proposal is tested by the approximate likelihood based model. The full likelihood based posterior computation will be conducted only if the proposal passes the first stage screening. Furthermore, this method is adopted into the consensus Monte Carlo framework. Methods are illustrated on two large datasets.

[1]  Arnaud Doucet,et al.  Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach , 2014, ICML.

[2]  Max Welling,et al.  Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget , 2013, ICML 2014.

[3]  Yalchin Efendiev,et al.  Bayesian Uncertainty Quantification for Subsurface Inversion Using a Multiscale Hierarchical Model , 2014, Technometrics.

[4]  Peter D. Hoff,et al.  Fast Inference for the Latent Space Network Model Using a Case-Control Approximate Likelihood , 2012, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[5]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[6]  Edward I. George,et al.  Bayes and big data: the consensus Monte Carlo algorithm , 2016, Big Data and Information Theory.

[7]  Paulo Cortez,et al.  A data-driven approach to predict the success of bank telemarketing , 2014, Decis. Support Syst..

[8]  C. Fox,et al.  Markov chain Monte Carlo Using an Approximation , 2005 .

[9]  Dave Higdon,et al.  A Bayesian approach to characterizing uncertainty in inverse problems using coarse and fine-scale information , 2002, IEEE Trans. Signal Process..