Identifying small subsets of agents for behavior tracking and abnormal event detection in dynamic networks

For very large dynamic networks, monitoring the behavior of a subset of agents provides an efficient framework for detecting changes in network topology. For example, in mobile caller networks with millions of subscribers, we would like to monitor the dynamics of the smallest possible set of subscribers and still be able to infer abnormal events that occur over the entire network. In general, we assume that the temporal behavior of a network agent is captured by a (local) dynamic state, which may reflect either a physical property such as the number of connections or an abstract quantity such as opinions or beliefs. Further, assuming coupled linear inter-agent dynamics in which the local agent states evolve as weighted linear combinations of the neighboring agents' states, we focus on tracking network-wide agent dynamics. Due to the large-scale nature of the problem, directly monitoring data streams of the state dynamics for every individual agent is infeasible. To address this issue, we propose a method that identifies a relatively small subset of agents whose state streams enable us to reconstruct the dynamic state evolution of all the network agents at any given time and, simultaneously, detect agent departure events. Using structural properties of the coupled inter-agent dynamics, we provide an algorithm, which is polynomial in the number of agents, to identify a small subset of agents that ensures such network observability regardless of any agent leaving. In addition, we show how well-known tools in dynamic control systems may be useful for identifying abnormal events; in particular, we use a fault detection and isolation scheme to identify agent departures. Finally, we illustrate our method and algorithms in a small test network as a proof of concept.

[1]  Hisashi Kashima,et al.  Eigenspace-based anomaly detection in computer systems , 2004, KDD.

[2]  R. Kálmán Mathematical description of linear dynamical systems , 1963 .

[3]  Alan S. Willsky,et al.  A survey of design methods for failure detection in dynamic systems , 1976, Autom..

[4]  Horst Bunke,et al.  Computer Network Monitoring and Abnormal Event Detection Using Graph Matching and Multidimensional Scaling , 2006, Industrial Conference on Data Mining.

[5]  Soummya Kar,et al.  A structured systems approach for optimal actuator-sensor placement in linear time-invariant systems , 2013, 2013 American Control Conference.

[6]  Ching-tai Lin Structural controllability , 1974 .

[7]  Christian Commault,et al.  Generic properties and control of linear structured systems: a survey , 2003, Autom..

[8]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[9]  F. Fairman Introduction to dynamic systems: Theory, models and applications , 1979, Proceedings of the IEEE.

[10]  Brandon Pincombea,et al.  Anomaly Detection in Time Series of Graphs using ARMA Processes , 2007 .

[11]  Christos Faloutsos,et al.  oddball: Spotting Anomalies in Weighted Graphs , 2010, PAKDD.

[12]  David J. Marchette,et al.  Scan Statistics on Enron Graphs , 2005, Comput. Math. Organ. Theory.

[13]  M. Kraetzl,et al.  Detection of abnormal change in dynamic networks , 1999, 1999 Information, Decision and Control. Data and Information Fusion Symposium, Signal Processing and Communications Symposium and Decision and Control Symposium. Proceedings (Cat. No.99EX251).

[14]  Albert-László Barabási,et al.  Controllability of complex networks , 2011, Nature.

[15]  J.J. Gertler,et al.  Survey of model-based failure detection and isolation in complex plants , 1988, IEEE Control Systems Magazine.

[16]  Hector Garcia-Molina,et al.  Web graph similarity for anomaly detection , 2010, Journal of Internet Services and Applications.

[17]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[18]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[19]  Albert-László Barabási,et al.  Observability of complex systems , 2013, Proceedings of the National Academy of Sciences.

[20]  Antonio Pedro Aguiar,et al.  Minimum Robust Sensor Placement for Large Scale Linear Time-Invariant Systems: A Structured Systems Approach* , 2013, IFAC Proceedings Volumes.

[21]  Gábor Wiener,et al.  On finding spanning trees with few leaves , 2008, Inf. Process. Lett..