Visual analysis of large-scale network anomalies

The amount of information flowing across communication networks has rapidly increased. The highly dynamic and complex networks, represented as large graphs, make the analysis of such networks increasingly challenging. In this paper, we provide a brief overview of several useful visualization techniques for the analysis of spatiotemporal anomalies in large-scale networks. We make use of community-based similarity graphs (CSGs), temporal expansion model graphs (TEMGs), correlation graphs (CGs), high-dimension projection graphs (HDPGs), and topology-preserving compressed graphs (TPCGs). CSG is used to detect anomalies based on community membership changes rather than individual nodes and edges and therefore may be more tolerant to the highly dynamic nature of large networks. TEMG transforms network topologies into directed trees so that efficient search is more likely to be performed for anomalous changes in network behavior and routing topology in large dynamic networks. CG and HDPG are used to examine the complex relationship of data dimensions among graph nodes through transformation in a high-dimensional space. TPCG groups nodes with similar neighbor sets into mega-nodes, thus making graph visualization and analysis more scalable to large networks. All the methods target efficient large-graph anomaly visualization from different perspectives and together provide valuable insights.

[1]  Heidrun Schumann,et al.  Axes-based visualizations with radial layouts , 2004, SAC '04.

[2]  Guy Melançon,et al.  Multiscale visualization of small world networks , 2003, IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714).

[3]  Diane J. Cook,et al.  Graph-based anomaly detection , 2003, KDD '03.

[4]  Yehuda Koren,et al.  Graph Drawing by Stress Majorization , 2004, GD.

[5]  Tina Eliassi-Rad,et al.  Detecting Novel Discrepancies in Communication Networks , 2010, 2010 IEEE International Conference on Data Mining.

[6]  Tanja Falkowski,et al.  Community analysis in dynamic social networks , 2009 .

[7]  Jimeng Sun,et al.  HiMap: Adaptive visualization of large-scale online social networks , 2009, 2009 IEEE Pacific Visualization Symposium.

[8]  Christos Faloutsos,et al.  Graph mining: Laws, generators, and algorithms , 2006, CSUR.

[9]  John V. Carlis,et al.  Interactive visualization of serial periodic data , 1998, UIST '98.

[10]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[11]  Pak Chung Wong,et al.  A multi-level middle-out cross-zooming approach for large graph analytics , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[12]  Lawrence B. Holder,et al.  Graph-Based Data Mining , 2000, IEEE Intell. Syst..

[13]  Yehuda Koren,et al.  Topological fisheye views for visualizing large graphs , 2004, IEEE Transactions on Visualization and Computer Graphics.

[14]  T. J. Jankun-Kelly,et al.  Detecting flaws and intruders with visual data analysis , 2004, IEEE Computer Graphics and Applications.

[15]  Tamara Munzner,et al.  H3: laying out large directed graphs in 3D hyperbolic space , 1997, Proceedings of VIZ '97: Visualization Conference, Information Visualization Symposium and Parallel Rendering Symposium.

[16]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[17]  Daniel A. McFarland,et al.  Dynamic Network Visualization1 , 2005, American Journal of Sociology.

[18]  David Harel,et al.  Graph Drawing by High-Dimensional Embedding , 2002, J. Graph Algorithms Appl..

[19]  Yunhao Liu,et al.  Agnostic diagnosis: Discovering silent failures in wireless sensor networks , 2011, INFOCOM.

[20]  Eser Kandogan,et al.  Visualizing multi-dimensional clusters, trends, and outliers using star coordinates , 2001, KDD '01.

[21]  Yunhao Liu,et al.  Does Wireless Sensor Network Scale? A Measurement Study on GreenOrbs , 2011, IEEE Transactions on Parallel and Distributed Systems.

[22]  Arjan Kuijper,et al.  Visual Analysis of Large Graphs: State‐of‐the‐Art and Future Research Challenges , 2011, Eurographics.

[23]  Derek G. Corneil,et al.  The graph isomorphism disease , 1977, J. Graph Theory.

[24]  Michael Garland,et al.  On the Visualization of Social and other Scale-Free Networks , 2008, IEEE Transactions on Visualization and Computer Graphics.

[25]  W. Wallis,et al.  A Graph-Theoretic Approach to Enterprise Network Dynamics , 2006 .

[26]  Yehuda Koren,et al.  Topological Fisheye Views for Visualizing Large Graphs , 2005, IEEE Trans. Vis. Comput. Graph..

[27]  Daniel W. Archambault,et al.  Structural differences between two graphs through hierarchies , 2009, Graphics Interface.

[28]  Gautam Kumar,et al.  Visual Exploration of Complex Time-Varying Graphs , 2006, IEEE Transactions on Visualization and Computer Graphics.

[29]  Peter Eades,et al.  FADE: Graph Drawing, Clustering, and Visual Abstraction , 2000, GD.

[30]  Danah Boyd,et al.  Vizster: visualizing online social networks , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[31]  Horst Bunke,et al.  A Graph-Theoretic Approach to Enterprise Network Dynamics (Progress in Computer Science and Applied Logic (PCS)) , 2006 .

[32]  Georges G. Grinstein,et al.  DNA visual and analytic data mining , 1997 .

[33]  Daniel A. Keim,et al.  Information Visualization and Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[34]  Lise Getoor,et al.  Link mining: a survey , 2005, SKDD.

[35]  Stephen G. Kobourov,et al.  Journal of Graph Algorithms and Applications Grip: Graph Drawing with Intelligent Placement , 2022 .

[36]  David Harel,et al.  ACE: a fast multiscale eigenvectors computation for drawing huge graphs , 2002, IEEE Symposium on Information Visualization, 2002. INFOVIS 2002..

[37]  Ramana Rao,et al.  A focus+context technique based on hyperbolic geometry for visualizing large hierarchies , 1995, CHI '95.

[38]  Michael Garland,et al.  Rapid Multipole Graph Drawing on the GPU , 2009, Graph Drawing.

[39]  Daniel A. Keim,et al.  Spatiotemporal Analysis of Sensor Logs using Growth Ring Maps , 2009, IEEE Transactions on Visualization and Computer Graphics.

[40]  Aaron Striegel,et al.  Intelligent network management using graph differential anomaly visualization , 2012, 2012 IEEE Network Operations and Management Symposium.

[41]  Andreas Paepcke,et al.  Visual Analysis of Network Flow Data with Timelines and Event Plots , 2007, VizSEC.

[42]  Alfred Inselberg,et al.  The plane with parallel coordinates , 1985, The Visual Computer.

[43]  Martin Wattenberg,et al.  Centrality Based Visualization of Small World Graphs , 2008, Comput. Graph. Forum.

[44]  Wayne G. Lutters,et al.  Focusing on context in network traffic analysis , 2006, IEEE Computer Graphics and Applications.

[45]  James Abello,et al.  ASK-GraphView: A Large Scale Graph Visualization System , 2006, IEEE Transactions on Visualization and Computer Graphics.

[46]  Georges G. Grinstein,et al.  Dimensional anchors: a graphic primitive for multidimensional multivariate information visualizations , 1999, NPIVM '99.

[47]  Satoru Kawai,et al.  An Algorithm for Drawing General Undirected Graphs , 1989, Inf. Process. Lett..

[48]  Daniel A. Keim,et al.  Large-Scale Network Monitoring for Visual Analysis of Attacks , 2008, VizSEC.

[49]  Len Bass,et al.  User interface software , 1993 .

[50]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[51]  A KeimDaniel Information Visualization and Visual Data Mining , 2002 .

[52]  Denis Lalanne,et al.  Visual Analysis of Corporate Network Intelligence: Abstracting and Reasoning on Yesterdays for Acting Today , 2007, VizSEC.

[53]  Yuan He,et al.  SAVE: Sensor anomaly visualization engine , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[54]  Georges G. Grinstein,et al.  DNA visual and analytic data mining , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[55]  Aiko Pras,et al.  A Labeled Data Set for Flow-Based Intrusion Detection , 2009, IPOM.

[56]  Heidrun Schumann,et al.  Visualization of Time-Oriented Data , 2011, Human-Computer Interaction Series.

[57]  Pavel Minarík,et al.  NetFlow Data Visualization Based on Graphs , 2008, VizSEC.

[58]  Christos Faloutsos,et al.  oddball: Spotting Anomalies in Weighted Graphs , 2010, PAKDD.

[59]  Douglas Thain,et al.  Distributed computing in practice: the Condor experience , 2005, Concurr. Pract. Exp..

[60]  S. Stigler Francis Galton's Account of the Invention of Correlation , 1989 .

[61]  Vladimir Batagelj,et al.  Exploratory Social Network Analysis with Pajek , 2005 .

[62]  Martin Wattenberg,et al.  Visual exploration of multivariate graphs , 2006, CHI.

[63]  Miron Livny,et al.  Distributed computing in practice: the Condor experience: Research Articles , 2005 .

[64]  Danny Holten,et al.  Hierarchical Edge Bundles: Visualization of Adjacency Relations in Hierarchical Data , 2006, IEEE Transactions on Visualization and Computer Graphics.

[65]  Lawrence B. Holder,et al.  Anomaly detection in data represented as graphs , 2007, Intell. Data Anal..