Melange: creating a "functional" internet

Most implementations of critical Internet protocols are written in type-unsafe languages such as C or C++ and are regularly vulnerable to serious security and reliability problems. Type-safe languages eliminate many errors but are not used to due to the perceived performance overheads. We combine two techniques to eliminate this performance penalty in a practical fashion: strong static typing and generative meta-programming. Static typing eliminates run-time type information by checking safety at compile-time and minimises dynamic checks. Meta-programming uses a single specification to abstract the low-level code required to transmit and receive packets. Our domain-specific language, MPL, describes Internet packet protocols and compiles into fast, zero-copy code for both parsing and creating these packets. MPL is designed for implementing quirky Internet protocols ranging from the low-level: Ethernet, IPv4, ICMP and TCP; to the complex application-level: SSH, DNS and BGP; and even file-system protocols such as 9P. We report on fully-featured SSH and DNS servers constructed using MPL and our OCaml framework Melange, and measure greater throughput, lower latency, better flexibility and more succinct source code than their C equivalents OpenSSH and BIND. Our quantitative analysis shows that the benefits of MPL-generated code overcomes the additional overheads of automatic garbage collection and dynamic bounds checking. Qualitatively, the flexibility of our approach shows that dramatic optimisations are easily possible.

[1]  Peter Lee,et al.  A Network Protocol Stack in Standard ML , 2001, High. Order Symb. Comput..

[2]  Richard Sharp,et al.  Linear Types for Packet Processing , 2004, ESOP.

[3]  Andreas Podelski,et al.  Termination proofs for systems code , 2006, PLDI '06.

[4]  Mark Garland Hayden,et al.  The Ensemble System , 1998 .

[5]  Anil Madhavapeddy,et al.  Creating high-performance statically type-safe network applications , 2010 .

[6]  Shari Lawrence Pfleeger,et al.  Investigating the Influence of Formal Methods , 1997, Computer.

[7]  Anil Madhavapeddy,et al.  On the challenge of delivering high-performance, dependable, model-checked internet servers , 2005 .

[8]  Niels Provos,et al.  Preventing Privilege Escalation , 2003, USENIX Security Symposium.

[9]  Todd A. Proebsting,et al.  USC: A Universal Stub Compiler , 1994, SIGCOMM.

[10]  Eddie Kohler,et al.  A readable TCP in the Prolac protocol language , 1999, SIGCOMM '99.

[11]  Feng Zhou,et al.  Thirty Years Is Long Enough: Getting Beyond C , 2005, HotOS.

[12]  Ken Thompson,et al.  The use of name spaces in Plan 9 , 1993, OPSR.

[13]  Niels Provos,et al.  ScanSSH: Scanning the Internet for SSH Servers , 2001, LISA.

[14]  Richard Sharp,et al.  SPLAT: A Tool for Model-Checking and Dynamically-Enforcing Abstractions , 2005, SPIN.

[15]  Robert Gruber,et al.  PADS: a domain-specific language for processing ad hoc data , 2005, PLDI '05.

[16]  Jon Crowcroft,et al.  The main name system: an exercise in centralized computing , 2005, CCRV.

[17]  Todd A. Proebsting,et al.  USC: a universal stub compiler , 1994, SIGCOMM 1994.

[18]  Herbert Bos,et al.  FFPF: Fairly Fast Packet Filters , 2004, OSDI.

[19]  Frank Pfenning,et al.  Eliminating array bound checking through dependent types , 1998, PLDI.

[20]  Eddie Kohler,et al.  The Click modular router , 1999, SOSP.

[21]  Hsiao-Keng Jerry Chu,et al.  Zero-Copy TCP in Solaris , 1996, USENIX Annual Technical Conference.

[22]  James Cheney,et al.  Cyclone: A Safe Dialect of C , 2002, USENIX Annual Technical Conference, General Track.

[23]  Larry L. Peterson,et al.  Fbufs: a high-bandwidth cross-domain transfer facility , 1994, SOSP '93.

[24]  Steven McCanne,et al.  The BSD Packet Filter: A New Architecture for User-level Packet Capture , 1993, USENIX Winter.

[25]  Didier Rémy,et al.  Objective ML: a simple object-oriented extension of ML , 1997, POPL '97.

[26]  Timothy John Deegan,et al.  The main name system , 2006 .

[27]  Robert Tappan Morris,et al.  DNS performance and the effectiveness of caching , 2001, IMW '01.

[28]  Peter Sewell,et al.  Type-safe distributed programming for OCaml , 2006, ML '06.

[29]  Andrea C. Arpaci-Dusseau,et al.  Deploying Safe User-Level Network Services with icTCP , 2004, OSDI.

[30]  Larry L. Peterson,et al.  Making paths explicit in the Scout operating system , 1996, OSDI '96.

[31]  Satish Chandra,et al.  Packet types: abstract specification of network protocol messages , 2000 .

[32]  Ion Stoica,et al.  Implementing declarative overlays , 2005, SOSP '05.

[33]  George C. Necula,et al.  CCured: type-safe retrofitting of legacy code , 2002, POPL '02.

[34]  Edoardo Biagioni.  A structured TCP in standard ML. , 1994, SIGCOMM 1994.

[35]  Larry L. Peterson,et al.  binpac: a yacc for writing application protocol parsers , 2006, IMC '06.

[36]  Benjamin C. Pierce,et al.  Combinators for bi-directional tree transformations: a linguistic approach to the view update problem , 2005, POPL '05.

[37]  Damien Doligez,et al.  A concurrent, generational garbage collector for a multithreaded implementation of ML , 1993, POPL '93.