P4Tester: Efficient Runtime Rule Fault Detection for Programmable Data Planes

P4 and programmable data planes bring significant flexibility to network operation but are inevitably prone to various faults. Some faults, like P4 program bugs, can be verified statically, while some faults, like runtime rule faults, only happen to running network devices, and they are hardly possible to handle before deployment. Existing network testing systems can troubleshoot runtime rule faults via injecting probes, but are insufficient for programmable data planes due to large overheads or limited fault coverage. In this paper, we propose P4Tester, a new network testing system for troubleshooting runtime rule faults on programmable data planes. First, P4Tester proposes a new intermediate representation based on Binary Decision Diagram, which enables efficient probe generation for various P4-defined data plane functions. Second, P4Tester offers a new probe model that uses source routing to forward probes. This probe model largely reduces rule fault detection overheads, i.e. requiring only one server to generate probes for large networks and minimizing the number of probes. Moreover, this probe model can test all table rules in a network, achieving full fault coverage. Evaluation based on real-world data sets indicates that P4Tester can efficiently check all rules in programmable data planes, generate 59% fewer probes than ATPG and Pronto, be faster than ATPG by two orders of magnitude, and troubleshoot multiple rule faults within one second on BMv2 and Tofino.

[1]  Fernando Pedone,et al.  NetPaxos: consensus at network speed , 2015, SOSR.

[2]  Ariel Orda,et al.  dRMT: Disaggregated Programmable Switching , 2017, SIGCOMM.

[3]  Xiaozhou Li,et al.  NetChain: Scale-Free Sub-RTT Coordination , 2018, NSDI.

[4]  Walter Willinger,et al.  Sonata: Query-Driven Network Telemetry , 2017, ArXiv.

[5]  Costin Raiciu,et al.  Debugging P4 programs with vera , 2018, SIGCOMM.

[6]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[7]  Panos Kalnis,et al.  In-Network Computation is a Dumb Idea Whose Time Has Come , 2017, HotNets.

[8]  David Walker,et al.  CacheFlow: Dependency-Aware Rule-Caching for Software-Defined Networks , 2016, SOSR.

[9]  Jennifer Rexford,et al.  HULA: Scalable Load Balancing Using Programmable Data Planes , 2016, SOSR.

[10]  Minlan Yu,et al.  SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs , 2017, SIGCOMM.

[11]  Yu Zhou,et al.  P4DB: On-the-fly debugging of the programmable data plane , 2017, 2017 IEEE 25th International Conference on Network Protocols (ICNP).

[12]  Chen Qian,et al.  Pronto: Efficient Test Packet Generation for Dynamic Network Data Planes , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[13]  Kirill Levchenko,et al.  Uncovering Bugs in P4 Programs with Assertion-based Verification , 2018, SOSR.

[14]  Peter M. Athanas,et al.  p4pktgen: Automated Test Case Generation for P4 Programs , 2018, SOSR.

[15]  Hua Chen,et al.  Pingmesh: A Large-Scale System for Data Center Network Latency Measurement and Analysis , 2015, SIGCOMM.

[16]  George Varghese,et al.  P4: programming protocol-independent packet processors , 2013, CCRV.

[17]  Nate Foster,et al.  NetCache: Balancing Key-Value Stores with Fast In-Network Caching , 2017, SOSP.

[18]  Carl A. Sunshine,et al.  Source routing in computer networks , 1977, CCRV.

[19]  Bo Yang,et al.  Is every flow on the right track?: Inspect SDN forwarding with RuleScope , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[20]  Nick McKeown,et al.  I Know What Your Packet Did Last Hop: Using Packet Histories to Troubleshoot Networks , 2014, NSDI.

[21]  Dejan Kostic,et al.  ProboScope: Data Plane Probe Packet Generation , 2014 .

[22]  Nick McKeown,et al.  p4v: practical verification for programmable data planes , 2018, SIGCOMM.

[23]  Cheng Zhang,et al.  Fast testing network data plane with RuleChecker , 2017, 2017 IEEE 25th International Conference on Network Protocols (ICNP).

[24]  Ben Y. Zhao,et al.  Packet-Level Telemetry in Large Datacenter Networks , 2015, SIGCOMM.

[25]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[26]  Maciej Kuźniar,et al.  What You Need to Know About SDN Flow Tables , 2015, PAM.

[27]  Peng Zhang,et al.  Towards rule enforcement verification for software defined networks , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[28]  George Varghese,et al.  Forwarding metamorphosis: fast programmable match-action processing in hardware for SDN , 2013, SIGCOMM.

[29]  George Varghese,et al.  Automatic Test Packet Generation , 2012, IEEE/ACM Transactions on Networking.

[30]  Dejan Kostic,et al.  Monocle: dynamic, fine-grained data plane monitoring , 2015, CoNEXT.