NSDMiner: Automated discovery of Network Service Dependencies

Enterprise networks today host a wide variety of network services, which often depend on each other to provide and support network-based services and applications. Understanding such dependencies is essential for maintaining the well-being of an enterprise network and its applications, particularly in the presence of network attacks and failures. In a typical enterprise network, which is complex and dynamic in configuration, it is non-trivial to identify all these services and their dependencies. Several techniques have been developed to learn such dependencies automatically. However, they are either too complex to fine tune or cluttered with false positives and/or false negatives. In this paper, we propose a suite of novel techniques and develop a new tool named NSDMiner (which stands for Mining for Network Service Dependencies) to automatically discover the dependencies between network services from passively collected network traffic. NSDMiner is non-intrusive; it does not require any modification of existing software, or injection of network packets. More importantly, NSDMiner achieves higher accuracy than previous network-based approaches. Our experimental evaluation, which uses network traffic collected from our campus network, shows that NSDMiner outperforms the two best existing solutions significantly.

[1]  Uri Blumenthal,et al.  Classification and computation of dependencies for distributed management , 2000, Proceedings ISCC 2000. Fifth IEEE Symposium on Computers and Communications.

[2]  Ranveer Chandra,et al.  What's going on?: learning communication rules in edge networks , 2008, SIGCOMM '08.

[3]  Richard Mortier,et al.  Using Magpie for Request Extraction and Workload Modelling , 2004, OSDI.

[4]  Jaideep Chandrashekar,et al.  Macroscope: end-point approach to networked application dependency discovery , 2009, CoNEXT '09.

[5]  Spyros G. Denazis,et al.  Dependency Detection Using a Fuzzy Engine , 2007, DSOM.

[6]  Randy H. Katz,et al.  X-Trace: A Pervasive Network Tracing Framework , 2007, NSDI.

[7]  Aaron B. Brown,et al.  An active approach to characterizing dynamic dependencies for problem determination in a distributed environment , 2001, 2001 IEEE/IFIP International Symposium on Integrated Network Management Proceedings. Integrated Network Management VII. Integrated Management Strategies for the New Millennium (Cat. No.01EX470).

[8]  Paramvir Bahl,et al.  Towards highly reliable enterprise network services via inference of multi-level dependencies , 2007, SIGCOMM '07.

[9]  Alexander Keller,et al.  Managing application services over service provider networks: architecture and dependency analysis , 2000, NOMS 2000. 2000 IEEE/IFIP Network Operations and Management Symposium 'The Networked Planet: Management Beyond 2000' (Cat. No.00CB37074).

[10]  Xu Chen,et al.  Automating Network Application Dependency Discovery: Experiences, Limitations, and New Solutions , 2008, OSDI.

[11]  Paramvir Bahl,et al.  Discovering Dependencies for Network Management , 2006, HotNets.

[12]  David A. Patterson,et al.  Path-Based Failure and Evolution Management , 2004, NSDI.