Nonparametric self-exciting models for computer network traffic

Connectivity patterns between nodes in a computer network can be interpreted and modelled as point processes where events in a process indicate connections being established for data to be sent along that edge. A model of normal connectivity behaviour can be constructed for each edge in a network by identifying key network user features such as seasonality or self-exciting behaviour, since events typically arise in bursts at particular times of day which may be peculiar to that edge. When monitoring a computer network in real time, unusual patterns of activity against the model of normality could indicate the presence of a malicious actor. A flexible, novel, nonparametric model for the excitation function of a Wold process is proposed for modelling the conditional intensities of network edges. This approach is shown to outperform standard seasonality and self-excitation models in predicting network connections, achieving well-calibrated predictions for event data collected from the computer networks of both Imperial College and Los Alamos National Laboratory.

[1]  Don X. Sun,et al.  Estimating Millions of Dynamic Timing Patterns in Real Time , 2001 .

[2]  Chuanhai Liu,et al.  Adaptive Thresholds , 2006 .

[3]  J. Naus,et al.  Scan Statistics , 2014, Encyclopedia of Social Network Analysis and Mining.

[4]  Niao He,et al.  Nonparametric Hawkes Processes: Online Estimation and Generalization Bounds , 2018, 1801.08273.

[5]  F. Massey The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[6]  Emery N. Brown,et al.  The Time-Rescaling Theorem and Its Application to Neural Spike Train Data Analysis , 2002, Neural Computation.

[7]  P. Fearnhead,et al.  Optimal detection of changepoints with a linear computational cost , 2011, 1101.1438.

[8]  Joshua Neil,et al.  Detecting Localised Anomalous Behaviour in a Computer Network , 2014, IDA.

[9]  M. T. Boswell Estimating and Testing Trend in a Stochastic Process of Poisson Type , 1966 .

[10]  Curtis B. Storlie,et al.  Scan Statistics for the Online Detection of Locally Anomalous Subgraphs , 2013, Technometrics.

[11]  K. Giesecke,et al.  Exploring the Sources of Default Clustering , 2017, Journal of Financial Economics.

[12]  Patrick Rubin-Delanchy,et al.  Network-wide anomaly detection via the Dirichlet process , 2016, 2016 IEEE Conference on Intelligence and Security Informatics (ISI).

[13]  OSÉ,et al.  Estimating Millions of Dynamic Timing Patterns in Real-Time , 2001 .

[14]  Nikhil Kumar Singh,et al.  An Approach to Understand the End User Behavior through Log Analysis , 2010 .

[15]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[16]  Scott W. Linderman,et al.  Discovering Latent Network Structure in Point Process Data , 2014, ICML.

[17]  Melissa J. Turcotte,et al.  Detecting Periodic Subsequences in Cyber Security Data , 2017, 2017 European Intelligence and Security Informatics Conference (EISIC).

[18]  Edsel A Peña,et al.  Randomised P-values and nonparametric procedures in multiple testing , 2011, Journal of nonparametric statistics.

[19]  Kun Zhang,et al.  Learning Network of Multivariate Hawkes Processes: A Time Series Approach , 2016, UAI.

[20]  H. Wold,et al.  On Prediction in Stationary Time Series , 1948 .

[21]  Jaideep Srivastava,et al.  A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection , 2003, SDM.

[22]  T. Ozaki Maximum likelihood estimation of Hawkes' self-exciting point processes , 1979 .

[23]  A. Hawkes Point Spectra of Some Mutually Exciting Point Processes , 1971 .

[24]  Alexander D. Kent,et al.  Unified Host and Network Data Set , 2017, Security Science and Technology.

[25]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[26]  Kang G. Shin,et al.  Hop-Count Filtering : An Effective Defense Against Spoofed Traffic , 2003 .

[27]  Yosihiko Ogata,et al.  Statistical Models for Earthquake Occurrences and Residual Analysis for Point Processes , 1988 .

[28]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[29]  Alexander D. Kent,et al.  Modelling user behaviour in a network using computer event logs , 2016 .

[30]  U. Mitra,et al.  Detection of low-rate attacks in computer networks , 2008, IEEE INFOCOM Workshops 2008.

[31]  Hongyuan Zha,et al.  Learning Granger Causality for Hawkes Processes , 2016, ICML.

[32]  Andrea L. Bertozzi,et al.  Modeling E-mail Networks and Inferring Leadership Using Self-Exciting Point Processes , 2016 .