Adaptive Monitoring: Application of Probing to Adapt Passive Monitoring

Availability of good quality monitoring data is a vital need for management of today’s data centers. However, effective use of monitoring tools demands an understanding of the monitoring requirements that system administrators most often lack. Instead of a well-defined process of defining a monitoring strategy, system administrators adopt a manual and intuition-based approach. In this paper, we propose to replace the ad-hoc, manual, intuition-based approach with a more systematic, automated, and analytics-based approach for system monitoring. We propose an adaptive monitoring framework where end-to-end probing-based solutions are used to adapt the at-a-point monitoring tools. We present a systematic framework to use probes for adjusting monitoring levels. We present algorithms to select and analyze probes and to dynamically adapt the monitoring policies based on probe analysis. We demonstrate the effectiveness of the proposed solution using real-world examples as well as simulations.

[1]  James Won-Ki Hong,et al.  The Architecture of NG-MON: A Passive Network Monitoring System for High-Speed IP Networks , 2002, DSOM.

[2]  Ling Huang,et al.  Communication-Efficient Online Detection of Network-Wide Anomalies , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[3]  Ehab Al-Shaer,et al.  QoS Path Monitoring for Multicast Networks , 2002, Journal of Network and Systems Management.

[4]  Ehab Al-Shaer,et al.  Active integrated fault localization in communication networks , 2005, 2005 9th IFIP/IEEE International Symposium on Integrated Network Management, 2005. IM 2005..

[5]  Abhishek Kumar,et al.  Lightweight, High-Resolution Monitoring for Troubleshooting Production Systems , 2008, OSDI.

[6]  Ibrahim Matta,et al.  BRITE: an approach to universal topology generation , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[7]  A. L. Scherr,et al.  AN ANALYSIS OF TIME-SHARED COMPUTER SYSTEMS , 1965 .

[8]  Maitreya Natu,et al.  Application of adaptive probing for fault diagnosis in computer networks , 2008, NOMS 2008 - 2008 IEEE Network Operations and Management Symposium.

[9]  Herb Schwetman,et al.  CSIM: a C-based process-oriented simulation language , 1986, WSC '86.

[10]  John S. Heidemann,et al.  Trinocular: understanding internet reliability through adaptive probing , 2013, SIGCOMM.

[11]  Maitreya Natu,et al.  Adaptive monitoring: A framework to adapt passive monitoring using probing , 2012, 2012 8th international conference on network and service management (cnsm) and 2012 workshop on systems virtualiztion management (svm).

[12]  Ross Ihaka,et al.  Gentleman R: R: A language for data analysis and graphics , 1996 .

[13]  Luciano Paschoal Gaspary,et al.  Assessing transaction-based Internet applications performance through a passive network traffic monitoring approach , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[14]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[15]  Genady Grabarnik,et al.  Active Probing , 2002 .

[16]  Jie Gao,et al.  Approaches to building self healing systems using dependency analysis , 2004, 2004 IEEE/IFIP Network Operations and Management Symposium (IEEE Cat. No.04CH37507).

[17]  Rebecca N. Wright,et al.  The design space of probing algorithms for network-performance measurement , 2013, SIGMETRICS '13.

[18]  Qiang Zheng,et al.  Minimizing Probing Cost and Achieving Identifiability in Probe-Based Network Link Monitoring , 2013, IEEE Transactions on Computers.

[19]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[20]  Maitreya Natu,et al.  Probabilistic Fault Diagnosis Using Adaptive Probing , 2007, DSOM.

[21]  G. W. STEWARTt.  ON THE EARLY HISTORY OF THE SINGULAR VALUE DECOMPOSITION * , 2022 .

[22]  Lu Cheng,et al.  An efficient active probing approach based on the combination of online and offline strategies , 2010, 2010 International Conference on Network and Service Management.

[23]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[24]  John Heidemann,et al.  Detecting Internet Outages with Precise Active Probing ( extended ) , 2011 .

[25]  Liam Fallon,et al.  Adaptive terminal reporting for scalable service quality monitoring in large networks , 2011, 2011 7th International Conference on Network and Service Management.

[26]  Minlan Yu,et al.  Profiling Network Performance for Multi-tier Data Center Applications , 2011, NSDI.

[27]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.

[28]  Richard Wolski,et al.  Experiences with predicting resource performance on-line in computational grid settings , 2003, PERV.

[29]  Sisi Liu,et al.  Gateway selection in hybrid wireless networks through cooperative probing , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[30]  Cyrus Shahabi,et al.  Feature subset selection and feature ranking for multivariate time series , 2005, IEEE Transactions on Knowledge and Data Engineering.

[31]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.