Compressing Bayesian Networks: Swarm-Based Descent, Efficiency, and Posterior Accuracy

Local models in Bayesian networks (BNs) reduce space complexity, facilitate acquisition, and can improve inference efficiency. This work focuses on Non-Impeding Noisy-AND Tree (NIN-AND Tree or NAT) models whose merits include linear complexity, being based on simple causal interactions, expressiveness, and generality. We present a swarm-based constrained gradient descent for more efficient compression of BN CPTs (conditional probability tables) into NAT models. We show empirically that multiplicatively factoring NAT-modeled BNs allows significant speed up in inference for a reasonable range of sparse BN structures. We also show that such gain in efficiency only causes reasonable approximation errors in posterior marginals in NAT-modeled real world BNs.

[1]  Yang Xiang,et al.  Non-impeding noisy-AND tree causal models over multi-valued variables , 2012, Int. J. Approx. Reason..

[2]  Yang Xiang,et al.  Efficient probabilistic inference in Bayesian networks with multi-valued NIN-AND tree local models , 2017, Int. J. Approx. Reason..

[3]  Jesse Hoey,et al.  SPUDD: Stochastic Planning using Decision Diagrams , 1999, UAI.

[4]  Yang Xiang,et al.  NAT model–based compression of Bayesian network CPTs over multivalued variables , 2018, Comput. Intell..

[5]  Anders L. Madsen,et al.  LAZY Propagation: A Junction Tree Inference Algorithm Based on Lazy Evaluation , 1999, Artif. Intell..

[6]  John F. Lemmer,et al.  Recursive noisy OR - a rule for estimating complex probabilistic interactions , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Yang Xiang,et al.  Compression of Bayesian Networks with NIN-AND Tree Modeling , 2014, Probabilistic Graphical Models.

[8]  Max Henrion,et al.  Some Practical Issues in Constructing Belief Networks , 1987, UAI.

[9]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Jirí Vomlel,et al.  An Approximate Tensor-Based Inference Method Applied to the Game of Minesweeper , 2014, Probabilistic Graphical Models.

[11]  Yang Xiang,et al.  Acquisition of Causal Models for Local Distributions in Bayesian Networks , 2014, IEEE Transactions on Cybernetics.

[12]  Linda C. van der Gaag,et al.  An intercausal cancellation model for Bayesian-network engineering , 2015, Int. J. Approx. Reason..

[13]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Francisco Javier Díez,et al.  Parameter adjustment in Bayes networks. The generalized noisy OR-gate , 1993, UAI.

[16]  Marek J. Druzdzel,et al.  An Independence of Causal Interactions Model for Opposing Inuences , 2008 .