Compiling Path Queries

Measuring the flow of traffic along network paths is crucial for many management tasks, including traffic engineering, diagnosing congestion, and mitigating DDoS attacks. We introduce a declarative query language for efficient path-based traffic monitoring. Path queries are specified as regular expressions over predicates on packet locations and header values, with SQL-like "groupby" constructs for aggregating results anywhere along a path. A run-time system compiles queries into a deterministic finite automaton. The automaton's transition function is then partitioned, compiled into match-action rules, and distributed over the switches. Switches stamp packets with automaton states to track the progress towards fulfilling a query. Only when packets satisfy a query are the packets counted, sampled, or sent to collectors for further analysis. By processing queries in the data plane, users "pay as they go", as data-collection overhead is limited to exactly those packets that satisfy the query. We implemented our system on top of the Pyretic SDN controller and evaluated its performance on a campus topology. Our experiments indicate that the system can enable "interactive debugging"-- compiling multiple queries in a few seconds--while fitting rules comfortably in modern switch TCAMs and the automaton state into two bytes (e.g., a VLAN header).

[1]  Minlan Yu,et al.  Enforcing Network-Wide Policies in the Presence of Dynamic Middlebox Actions using FlowTags , 2014, NSDI.

[2]  Rodrigo Fonseca,et al.  Planck , 2014, SIGCOMM.

[3]  Olivier Bonaventure,et al.  IGen: Generation of router-level Internet topologies through network design heuristics , 2009, 2009 21st International Teletraffic Congress.

[4]  Ramesh Govindan,et al.  DREAM , 2014, SIGCOMM.

[5]  Ratul Mahajan,et al.  Measuring ISP topologies with Rocketfuel , 2004, IEEE/ACM Transactions on Networking.

[6]  Xu Chen,et al.  DECOR: DEClaritive network management and OpeRation , 2009, PRESTO '09.

[7]  Chen-Nee Chuah,et al.  ProgME: towards programmable network measurement , 2007, SIGCOMM 2007.

[8]  Anees Shaikh,et al.  Virtual network diagnosis as a service , 2013, SoCC.

[9]  Marina Thottan,et al.  Latency in Software Defined Networks: Measurements and Mitigation Techniques , 2015, SIGMETRICS.

[10]  Anna R. Karlin,et al.  Practical network support for IP traceback , 2000, SIGCOMM.

[11]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[12]  Scott Smith,et al.  Keeping Track of 70, 000+ Servers: The Akamai Query System , 2010, LISA.

[13]  Anja Feldmann,et al.  Deriving traffic demands for operational IP networks: methodology and experience , 2000, SIGCOMM.

[14]  BERNARD M. WAXMAN,et al.  Routing of multipoint connections , 1988, IEEE J. Sel. Areas Commun..

[15]  Brighten Godfrey,et al.  VeriFlow: verifying network-wide invariants in real time , 2012, HotSDN '12.

[16]  Christian E. Hopps,et al.  Analysis of an Equal-Cost Multi-Path Algorithm , 2000, RFC.

[17]  David Walker,et al.  Composing Software Defined Networks , 2013, NSDI.

[18]  Albert G. Greenberg,et al.  Ananta: cloud scale load balancing , 2013, SIGCOMM.

[19]  Russell J. Clark,et al.  SDX , 2014 .

[20]  Sujata Banerjee,et al.  DevoFlow: scaling flow management for high-performance networks , 2011, SIGCOMM.

[21]  Nate Foster,et al.  NetKAT: semantic foundations for networks , 2014, POPL.

[22]  Emin Gün Sirer,et al.  NetQuery: a knowledge plane for reasoning about network properties , 2010, CoNEXT '10 Student Workshop.

[23]  G. Weikum.  Querying the Internet with PIER , 2005 .

[24]  George Varghese,et al.  Forwarding metamorphosis: fast programmable match-action processing in hardware for SDN , 2013, SIGCOMM.

[25]  Minlan Yu,et al.  Profiling Network Performance for Multi-tier Data Center Applications , 2011, NSDI.

[26]  George Varghese,et al.  Compiling Packet Programs to Reconfigurable Switches , 2015, NSDI.

[27]  Alex C. Snoeren,et al.  High-fidelity switch models for software-defined network emulation , 2013, HotSDN '13.

[28]  David Walker,et al.  Compiling path queries in software-defined networks , 2014, HotSDN.

[29]  Fernando Pedone,et al.  Merlin: A Language for Provisioning Network Resources , 2014, CoNEXT.

[30]  Myungjin Lee,et al.  CherryPick: tracing packet trajectory in software-defined datacenter networks , 2015, SOSR.

[31]  Jürgen Falb,et al.  The Internet Protocol , 2005, The Industrial Information Technology Handbook.

[32]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[33]  Nick McKeown,et al.  I Know What Your Packet Did Last Hop: Using Packet Histories to Troubleshoot Networks , 2014, NSDI.

[34]  Chen-Nee Chuah,et al.  ProgME: Towards Programmable Network MEasurement , 2007, IEEE/ACM Transactions on Networking.

[35]  Minlan Yu,et al.  SIMPLE-fying middlebox policy enforcement using SDN , 2013, SIGCOMM.

[36]  Marco Canini,et al.  FatTire: declarative fault tolerance for software-defined networks , 2013, HotSDN '13.

[37]  Theodore Johnson,et al.  Gigascope: a stream database for network applications , 2003, SIGMOD '03.

[38]  David A. Maltz,et al.  Towards Systematic Design of Enterprise Networks , 2008, IEEE/ACM Transactions on Networking.

[39]  David Walker,et al.  Concurrent NetCore: from policies to pipelines , 2014, ICFP 2014.

[40]  Nick G. Duffield,et al.  Trajectory sampling for direct traffic observation , 2001, TNET.

[41]  Ben Y. Zhao,et al.  Packet-Level Telemetry in Large Datacenter Networks , 2015, SIGCOMM.

[42]  Navendu Jain,et al.  Understanding network failures in data centers: measurement, analysis, and implications , 2011, SIGCOMM.

[43]  Craig Partridge,et al.  Hash-based IP traceback , 2001, SIGCOMM.

[44]  Richard Wang,et al.  OpenFlow-Based Server Load Balancing Gone Wild , 2011, Hot-ICE.

[45]  Minlan Yu,et al.  Software Defined Traffic Measurement with OpenSketch , 2013, NSDI.

[46]  Loris D'Antoni,et al.  Minimization of symbolic automata , 2014, POPL.

[47]  Rolf Stadler,et al.  A bottom-up approach to real-time search in large networks and clouds , 2016, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.

[48]  George Varghese,et al.  Header Space Analysis: Static Checking for Networks , 2012, NSDI.

[49]  Qiang Xu,et al.  Enabling layer 2 pathlet tracing through context encoding in software-defined networking , 2014, HotSDN.

[50]  James R. Larus,et al.  Efficient path profiling , 1996, Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29.

[51]  Arjun Guha,et al.  A fast compiler for NetKAT , 2015, ICFP.

[52]  Minlan Yu,et al.  Identifying performance bottlenecks in CDNs through TCP-level monitoring , 2011, W-MUST '11.

[53]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[54]  Kevin Borders,et al.  Chimera: A Declarative Language for Streaming Network Traffic Analysis , 2012, USENIX Security Symposium.

[55]  David Walker,et al.  Frenetic: a network programming language , 2011, ICFP.

[56]  Anja Feldmann,et al.  NetScope: traffic engineering for IP networks , 2000, IEEE Netw..

[57]  Anirudh Sivaraman,et al.  In-band Network Telemetry via Programmable Dataplanes , 2015 .

[58]  Da Yu,et al.  Simon: scriptable interactive monitoring for SDNs , 2015, SOSR.

[59]  Matthew Caesar,et al.  Toward Interactive Debugging for ISP Networks , 2009, HotNets.

[60]  Albert G. Greenberg,et al.  Fast accurate computation of large-scale IP traffic matrices from link loads , 2003, SIGMETRICS '03.

[61]  Robert B. Miller,et al.  Response time in man-computer conversational transactions , 1899, AFIPS Fall Joint Computing Conference.