CT-NOR: Representing and Reasoning About Events in Continuous Time

We present a generative model for representing and reasoning about the relationships among events in continuous time. We apply the model to the domain of networked and distributed computing environments where we fit the parameters of the model from timestamp observations, and then use hypothesis testing to discover dependencies between the events and changes in behavior for monitoring and diagnosis. After introducing the model, we present an EM algorithm for fitting the parameters and then present the hypothesis testing approach for both dependence discovery and change-point detection. We validate the approach for both tasks using real data from a trace of network events at Microsoft Research Cambridge. Finally, we formalize the relationship between the proposed model and the noisy-or gate for cases when time can be discretized.

[1]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[2]  R. Durrett.  Probability: Theory and Examples , 1993 .

[3]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[4]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[5]  David Heckerman,et al.  Causal independence for probability assessment and inference using Bayesian networks , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[6]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[7]  John D. Storey.  A direct approach to false discovery rates , 2002 .

[8]  Daphne Koller,et al.  Learning Continuous Time Bayesian Networks , 2002, UAI.

[9]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[10]  Larry Wasserman,et al.  All of Statistics , 2004 .

[11]  Thore Graepel,et al.  Poisson-Networks: A Model for Structured Poisson Processes. , 2005 .

[12]  David Heckerman,et al.  Determining the Number of Non-Spurious Arcs in a Learned DAG Model: Investigation of a Bayesian and a Frequentist Approach , 2007, UAI.

[13]  Paramvir Bahl,et al.  Towards highly reliable enterprise network services via inference of multi-level dependencies , 2007, SIGCOMM.

[14]  Richard Mortier,et al.  Constellation: automated discovery of service and host dependencies in networked systems , 2008 .