Como: An open infrastructure for network monitoring-research agenda

Our idea is to define a per-query accuracy metric and maximize the aggregate accuracy of all query results given the available resources. We argue that in network monitoring it is always possible to regulate the accuracy of a query (that will include the measurement error as well as the timeliness of the results). For a given set of queries we then need to define an aggregate utility function and be able to exercise a fine-grained control over it. To overcome this challenge we need to address two orthogonal sub-challenges. The first has to do with allowing users to post any query to the systems in a secure fashion. The second is about the definition and regulation of queries’ accuracy in a distributed monitoring infrastructure. We now describe the two sub-challenges in detail and introduce our ideas to overcome them.

[1]  T. V. Lakshman,et al.  High-speed policy-based packet forwarding using efficient multi-dimensional range matching , 1998, SIGCOMM '98.

[2]  Anees Shaikh,et al.  Load-sensitive routing of long-lived IP flows , 1999, SIGCOMM '99.

[3]  Lixia Zhang,et al.  On the placement of Internet instrumentation , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[4]  EDDIE KOHLER,et al.  The click modular router , 2000, TOCS.

[5]  Cristian Estan,et al.  New directions in traffic measurement and accounting , 2001, IMW '01.

[6]  Zhi-Li Zhang,et al.  Adaptive packet sampling for flow volume measurement , 2002, CCRV.

[7]  James Cheney,et al.  Cyclone: A Safe Dialect of C , 2002, USENIX Annual Technical Conference, General Track.

[8]  Jeffrey C. Mogul.  Trace anonymization misses the point , 2002, WWW 2002.

[9]  Michael Stonebraker,et al.  Monitoring Streams - A New Class of Data Management Applications , 2002, VLDB.

[10]  Carsten Lund,et al.  Properties and prediction of flow statistics from sampled packet streams , 2002, IMW '02.

[11]  Erik D. Demaine,et al.  Frequency Estimation of Internet Packet Streams with Limited Space , 2002, ESA.

[12]  Srinivasan Seshan,et al.  IrisNet: An Architecture for a Worldwide Sensor Web , 2003, IEEE Pervasive Comput..

[13]  Young-Jin Kim,et al.  Multi-dimensional range queries in sensor networks , 2003, SenSys '03.

[14]  Konstantina Papagiannaki,et al.  Long-term forecasting of Internet backbone traffic: observations and initial models , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[15]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.

[16]  Alefiya Hussain,et al.  Identification of Repeated Attacks Using Network Traffic Forensics , 2003 .

[17]  David Moore,et al.  Internet quarantine: requirements for containing self-propagating code , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[18]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[19]  George Varghese,et al.  Automatically inferring patterns of resource consumption in network traffic , 2003, SIGCOMM '03.

[20]  Carsten Lund,et al.  Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure , 2003, IMC '03.

[21]  Scott Shenker,et al.  Querying the Internet with PIER , 2003, VLDB.

[22]  Athina Markopoulou,et al.  Characterization of failures in an IP backbone , 2004, IEEE INFOCOM 2004.

[23]  B. Karp,et al.  Autograph: Toward Automated, Distributed Worm Signature Detection , 2004, USENIX Security Symposium.

[24]  Donald F. Towsley,et al.  Inferring TCP connection characteristics through passive measurements , 2004, IEEE INFOCOM 2004.

[25]  Moses Charikar,et al.  Finding frequent items in data streams , 2004, Theor. Comput. Sci..

[26]  Sriram Ramabhadran,et al.  Prefix Hash Tree An Indexing Data Structure over Distributed Hash Tables , 2004, PODC 2004.

[27]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM '04.

[28]  Konstantina Papagiannaki,et al.  Impact of flow dynamics on traffic engineering design principles , 2004, IEEE INFOCOM 2004.

[29]  Christophe Diot,et al.  The CoMo white paper , 2004 .

[30]  Vern Paxson,et al.  The top speed of flash worms , 2004, WORM '04.

[31]  Richard Sharp,et al.  Linear Types for Packet Processing , 2004, ESOP.

[32]  Hector Garcia-Molina,et al.  Online Balancing of Range-Partitioned Data with Applications to Peer-to-Peer Systems , 2004, VLDB.

[33]  Larry L. Peterson,et al.  Sophia: an Information Plane for networked systems , 2004, Comput. Commun. Rev..

[34]  David R. Karger,et al.  Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems , 2004, IPTPS.

[35]  Graham Cormode,et al.  What's hot and what's not: tracking most frequent items dynamically , 2003, TODS.

[36]  Carsten Lund,et al.  Estimating flow distributions from sampled flow statistics , 2003, SIGCOMM '03.

[37]  Donald F. Towsley,et al.  Locating network monitors: complexity, heuristics, and coverage , 2005, INFOCOM.

[38]  Chadi Barakat,et al.  Ranking flows from sampled traffic , 2005, CoNEXT '05.

[39]  Ramesh Govindan,et al.  Advanced Indexing Techniques for Wide-Area Network Monitoring , 2005, 21st International Conference on Data Engineering Workshops (ICDEW'05).

[40]  Rajeev Rastogi,et al.  Robust Monitoring of Link Delays and Faults in IP Networks , 2003, IEEE/ACM Transactions on Networking.

[41]  Christophe Diot,et al.  Reformulating the Monitor Placement Problem: Optimal Network-Wide Sampling , 2006 .

[42]  Nicolas Hohn,et al.  Inverting sampled traffic , 2003, IMC '03.