Indirect causes, dependencies and causality in Bayesian networks

Modeling causal dependencies in complex or time-dependent domains often demands cyclic dependencies. Such cycles arise from local points of views on dependencies where no singular causality is identifiable, i.e., roles of causes and effects are not universally identifiable. Modeling causation instead of correlation is of utmost importance, which is why Bayesian networks are frequently used to reason under uncertainty. Bayesian networks are probabilistic graphical models and allow for a causal modeling approach with locally specifiable and interpretable parameters, but are not defined for cyclic graph structures. If Bayesian networks are to be used for modeling uncertainties, cycles are eliminated with dynamic Bayesian networks, eliminating cycles over time. However, we show that eliminating cycles over time eliminates an anticipation of indirect influences as well, and enforces an infinitesimal resolution of time. Without a “causal design,” i.e., without representing direct and indirect causes appropriately, such networks return spurious results. In particular, the main novel contributions of this thesis can be summarized as follows. By considering specific properties of local conditional probability distributions, we show that a novel form of probabilistic graphical models rapidly adapts itself to a specific context at every timestep and, by that, correctly anticipates indirect influences under an unrestricted time granularity, even if cyclic dependencies arise. We show that this novel form of probabilistic graphical models follows familiar Bayesian networks’ syntax and semantics, despite being based on cyclic graphs. Throughout this thesis, we show that no external reasoning frameworks are required, no novel calculus needs be introduced, no computational overhead for solving common inference-, queryand learningproblems is introduced, and that familiar algorithmic schemes remain applicable. We feel confident to say that Bayesian networks and dynamic Bayesian networks can be based on cyclic graphs. In effect, we show, for the very first time, that a novel dynamic probabilistic graphical model is an intrinsic representation of a full joint probability distribution over multiple full joint probability distributions.

[1]  Paulo Cesar G. da Costa,et al.  Using a Semantic Approach to Cyber Impact Assessment , 2013, STIDS.

[2]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[3]  Andrew W. Appel,et al.  MulVAL: A Logic-based Network Security Analyzer , 2005, USENIX Security Symposium.

[4]  Nir Friedman,et al.  Learning Belief Networks in the Presence of Missing Values and Hidden Variables , 1997, ICML.

[5]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.

[6]  Fredrik Gustafsson,et al.  On Resampling Algorithms for Particle Filters , 2006, 2006 IEEE Nonlinear Statistical Signal Processing Workshop.

[7]  Pedro M. Domingos,et al.  Relational Dynamic Bayesian Networks , 2005, J. Artif. Intell. Res..

[8]  Sabine Glesner,et al.  Constructing Flexible Dynamic Belief Networks from First-Order Probalistic Knowledge Bases , 1995, ECSQARU.

[9]  Ralf Möller,et al.  Semantic Normalization and Merging of Business Dependency Models Preliminary Version , 2016 .

[10]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[11]  Zoubin Ghahramani,et al.  An Introduction to Hidden Markov Models and Bayesian Networks , 2001, Int. J. Pattern Recognit. Artif. Intell..

[12]  Ralf Möller,et al.  Context- and bias-free probabilistic mission impact assessment , 2017, Comput. Secur..

[13]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[14]  Peng Liu,et al.  Using Bayesian networks for cyber security analysis , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[15]  Judea Pearl,et al.  Reasoning with Cause and Effect , 1999, IJCAI.

[16]  Fabio Gagliardi Cozman,et al.  Axiomatizing Noisy-OR , 2004, ECAI.

[17]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[18]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[19]  Nevin Lianwen Zhang,et al.  Exploiting Contextual Independence In Probabilistic Inference , 2011, J. Artif. Intell. Res..

[20]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[21]  Luis Enrique Sucar,et al.  Bayesian Networks for Reliability Analysis of Complex Systems , 1998, IBERAMIA.

[22]  Hector J. Levesque,et al.  The Winograd Schema Challenge , 2011, AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning.

[23]  Nando de Freitas,et al.  Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks , 2000, UAI.

[24]  M. Drton Discrete chain graph models , 2009, 0909.0843.

[25]  Max Henrion,et al.  Practical issues in constructing a Bayes belief network , 1987, Int. J. Approx. Reason..

[26]  Sushil Jajodia,et al.  An Attack Graph-Based Probabilistic Security Metric , 2008, DBSec.

[27]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[28]  Ralf Möller,et al.  Indirect Causes in Dynamic Bayesian Networks Revisited , 2015, IJCAI.

[29]  P Haddawy,et al.  Clinical simulation using context-sensitive temporal probability models. , 1995, Proceedings. Symposium on Computer Applications in Medical Care.

[30]  Ralph Langner To Kill a Centrifuge A Technical Analysis of What Stuxnet ’ s Creators Tried to Achieve , 2013 .

[31]  Luc De Raedt,et al.  Inference and learning in probabilistic logic programs using weighted Boolean formulas , 2013, Theory and Practice of Logic Programming.

[32]  Yu Liu,et al.  Network vulnerability assessment using Bayesian networks , 2005, SPIE Defense + Commercial Sensing.

[33]  Luis M. de Campos,et al.  Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs , 2011, J. Artif. Intell. Res..

[34]  Somesh Jha,et al.  Two formal analyses of attack graphs , 2002, Proceedings 15th IEEE Computer Security Foundations Workshop. CSFW-15.

[35]  Le Song,et al.  Time-Varying Dynamic Bayesian Networks , 2009, NIPS.

[36]  Manfred Jaeger,et al.  Complex Probabilistic Modeling with Recursive Relational Bayesian Networks , 2001, Annals of Mathematics and Artificial Intelligence.

[37]  Ralf Möller,et al.  Probabilistic Mission Impact Assessment based on Widespread Local Events , 2015 .

[38]  N. J. A. Sloane,et al.  The On-Line Encyclopedia of Integer Sequences , 2003, Electron. J. Comb..

[39]  Stuart J. Russell,et al.  Approximate Inference for Infinite Contingent Bayesian Networks , 2005, AISTATS.

[40]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[41]  David Heckerman,et al.  Causal independence for probability assessment and inference using Bayesian networks , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[42]  Peter Martini,et al.  Graph based Metrics for Intrusion Response Measures in Computer Networks , 2007 .

[43]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[44]  Alexander J. Hartemink,et al.  Non-stationary dynamic Bayesian networks , 2008, NIPS.

[45]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[46]  Daphne Koller,et al.  Continuous Time Bayesian Networks , 2012, UAI.

[47]  Avi Pfeffer,et al.  Asynchronous Dynamic Bayesian Networks , 2005, UAI.

[48]  Peter Haddawy,et al.  A Theoretical Framework for Context-Sensitive Temporal Probability Model Construction with Application to Plan Projection , 1995, UAI.

[49]  Barton C. Massey Fast perfect weighted resampling , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[50]  John R. Goodall,et al.  Mission Impact of Cyber Events: Scenarios and Ontology to Express the Relationships between Cyber Assets, Missions, and Users , 2009 .

[51]  Gregory S. Parnell,et al.  Mission Oriented Risk and Design Analysis of Critical Information Systems , 2005 .

[52]  Ralf Möller,et al.  Exploiting Innocuousness in Bayesian Networks , 2015, Australasian Conference on Artificial Intelligence.

[53]  Tomoyuki Higuchi,et al.  Estimating time-dependent gene networks from time series microarray data by dynamic linear models with Markov switching , 2005, 2005 IEEE Computational Systems Bioinformatics Conference (CSB'05).

[54]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[55]  Ralf Möller,et al.  Probabilistic Mission Defense and Assurance , 2016 .

[56]  Sampath Srinivas,et al.  A Generalization of the Noisy-Or Model , 1993, UAI.

[57]  Nevin Lianwen Zhang,et al.  Exploiting Causal Independence in Bayesian Network Inference , 1996, J. Artif. Intell. Res..

[58]  Nora Cuppens-Boulahia,et al.  Cost Evaluation for Intrusion Response Using Dependency Graphs , 2009, 2009 International Conference on Network and Service Security.

[59]  Florent Masseglia,et al.  Time Aware Mining of Itemsets , 2008, 2008 15th International Symposium on Temporal Representation and Reasoning.

[60]  Hervé Debar,et al.  Towards an Automated and Dynamic Risk Management Response System , 2016, NordSec.

[61]  Marek J. Druzdzel,et al.  Probabilistic Independence of Causal Influences , 2006, Probabilistic Graphical Models.

[62]  David Heckerman,et al.  Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..

[63]  Jennifer Chu-Carroll,et al.  Building Watson: An Overview of the DeepQA Project , 2010, AI Mag..

[64]  S. Musman,et al.  Evaluating the Impact of Cyber Attacks on Missions , 2010 .

[65]  Daphne Koller,et al.  Learning Continuous Time Bayesian Networks , 2002, UAI.

[66]  John R. Goodall,et al.  Camus: Automatically mapping Cyber Assets to Missions and Users , 2009, MILCOM 2009 - 2009 IEEE Military Communications Conference.

[67]  Teodor Sommestad,et al.  Intrusion detection and the role of the system administrator , 2012, Inf. Manag. Comput. Secur..

[68]  Peter Haddawy,et al.  Answering Queries from Context-Sensitive Probabilistic Knowledge Bases (cid:3) , 1996 .

[69]  Zoubin Ghahramani,et al.  Learning Dynamic Bayesian Networks , 1997, Summer School on Neural Networks.

[70]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[71]  Gabriel Jakobson,et al.  Mission cyber security situation assessment using impact dependency graphs , 2011, 14th International Conference on Information Fusion.

[72]  Hervé Debar,et al.  Selection of Mitigation Actions Based on Financial and Operational Impact Assessments , 2016, 2016 11th International Conference on Availability, Reliability and Security (ARES).