Brief survey on computational solutions for Bayesian inference

In this paper, we present a brief review of research work attempting to tackle the issue of tractability in Bayesian inference, including an analysis of the applicability and trade-offs of each proposed solution. In recent years, the Bayesian approach has become increasingly popular, endowing autonomous systems with the ability to deal with uncertainty and incompleteness. However, these systems are also expected to be efficient, while Bayesian inference in general is known to be an NP-hard problem, making it paramount to develop approaches dealing with this complexity in order to allow the implementation of usable Bayesian solutions. Novel computational paradigms and also major developments in massively parallel computation technologies, such as multi-core processors, GPUs and FPGAs, provide us with an inkling of the roadmap in Bayesian computation for upcoming years.

[1]  Ross D. Shachter,et al.  Symbolic Probabilistic Inference in Belief Networks , 1990, AAAI.

[2]  J. Dias,et al.  Synthesis of Bayesian Machines On FPGAs Using Stochastic Arithmetic , 2014 .

[3]  Frank Jensen,et al.  Approximations in Bayesian Belief Universe for Knowledge Based Systems , 2013, UAI 1990.

[4]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[5]  Ross D. Shachter.  Intelligent Probabilistic Inference , 1985, UAI.

[6]  Kamel Mekhnacha,et al.  Bayesian Programming , 2013 .

[7]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[8]  Charu C. Aggarwal,et al.  Distributed Adaptive Importance Sampling on graphical models using MapReduce , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[9]  Avi Pfeffer,et al.  The Design and Implementation of IBAL: A General-Purpose Probabilistic Language , 2005 .

[10]  Viktor K. Prasanna,et al.  Parallel exact inference on the Cell Broadband Engine processor , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[11]  John Wawrzynek,et al.  High-throughput bayesian computing machine with reconfigurable hardware , 2010, FPGA '10.

[12]  Linda Smail,et al.  Successive Restrictions Algorithm in Bayesian Networks , 2005, IDA.

[13]  Markus Stumptner,et al.  A Survey of Directed Entity-Relation--Based First-Order Probabilistic Languages , 2014, ACM Comput. Surv..

[14]  John Wawrzynek,et al.  MARC: A Many-Core Approach to Reconfigurable Computing , 2010, 2010 International Conference on Reconfigurable Computing and FPGAs.

[15]  Gregory F. Cooper,et al.  NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge. , 1984 .

[16]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[17]  Benjamin Vigoda,et al.  Analog Logic: Continuous-Time Analog Circuits for Statistical Signal Processing , 2003 .

[18]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[19]  Viktor K. Prasanna,et al.  Scalable Node-Level Computation Kernels for Parallel Exact Inference , 2010, IEEE Transactions on Computers.

[20]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[21]  N. Ma.  EXACT INFERENCE ON MANYCORE PROCESSORS USING POINTER JUMPING , 2010 .

[22]  Viktor K. Prasanna,et al.  Junction tree decomposition for parallel exact inference , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[23]  Patrick Robertson,et al.  Comparison of exact static and dynamic Bayesian context inference methods for activity recognition , 2010, 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[24]  Jorge Dias,et al.  Probabilistic Approaches to Robotic Perception , 2014, Springer Tracts in Advanced Robotics.

[25]  Umut A. Acar,et al.  Adaptive Exact Inference in Graphical Models , 2011, J. Mach. Learn. Res..

[26]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[28]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[29]  Andrew McCallum,et al.  FACTORIE: Efficient Probabilistic Programming for Relational Factor Graphs via Imperative Declarations of Structure, Inference and Learning , 2008 .

[30]  David Poole,et al.  The use of conflicts in searching Bayesian networks , 1993, UAI.

[31]  Viktor K. Prasanna,et al.  Scalable parallel implementation of exact inference in Bayesian networks , 2006, 12th International Conference on Parallel and Distributed Systems - (ICPADS'06).

[32]  Viktor K. Prasanna,et al.  Scalable Parallel Implementation of Bayesian Network to Junction Tree Conversion for Exact Inference , 2006, 2006 18th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD'06).

[33]  Taisuke Sato,et al.  A glimpse of symbolic-statistical modeling by PRISM , 2008, Journal of Intelligent Information Systems.

[34]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[35]  Michael I. Jordan,et al.  Variational Probabilistic Inference and the QMR-DT Network , 2011, J. Artif. Intell. Res..

[36]  A. Pfeffer,et al.  Figaro : An Object-Oriented Probabilistic Programming Language , 2009 .

[37]  William H. Hsu,et al.  A Survey of Algorithms for Real-Time Bayesian Network Inference , 2002 .

[38]  Judea Pearl,et al.  A Computational Model for Causal and Diagnostic Reasoning in Inference Systems , 1983, IJCAI.

[39]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[40]  Pablo Lanillos,et al.  Fast Exact Bayesian Inference for High-Dimensional Models , 2015 .

[41]  John Wawrzynek,et al.  Exploring Many-Core Design Templates for FPGAs and ASICs , 2012, Int. J. Reconfigurable Comput..

[42]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[43]  Mark S. Boddy,et al.  Anytime Problem Solving Using Dynamic Programming , 1991, AAAI.

[44]  Viktor K. Prasanna,et al.  Parallel Exact Inference on a CPU-GPGPU Heterogenous System , 2010, 2010 39th International Conference on Parallel Processing.

[45]  Ole J. Mengshoel,et al.  Belief Propagation by Message Passing in Junction Trees: Computing Each Message Faster Using GPU Parallelization , 2011, UAI.

[46]  Viktor K. Prasanna,et al.  Node Level Primitives for Parallel Exact Inference , 2007, 19th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD'07).

[47]  Viktor K. Prasanna,et al.  Parallel Evidence Propagation on Multicore Processors , 2009, PaCT.

[48]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[49]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[50]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[51]  Linda Smail,et al.  Exact and approximate inference in ProBT , 2007, Rev. d'Intelligence Artif..

[52]  Joshua B. Tenenbaum,et al.  Natively probabilistic computation , 2009 .

[53]  Fabio Gagliardi Cozman,et al.  Anytime anyspace probabilistic inference , 2004, Int. J. Approx. Reason..