Prognosis: closed-box analysis of network protocol implementations

We present Prognosis, a framework offering automated closed-box learning and analysis of models of network protocol implementations. Prognosis can learn models that vary in abstraction level from simple deterministic automata to models containing data operations, such as register updates, and can be used to unlock a variety of analysis techniques -- model checking temporal properties, computing differences between models of two implementations of the same protocol, or improving testing via model-based test generation. Prognosis is modular and easily adaptable to different protocols (e.g. TCP and QUIC) and their implementations. We use Prognosis to learn models of (parts of) three QUIC implementations -- Quiche (Cloudflare), Google QUIC, and Facebook mvfst -- and use these models to analyse the differences between the various implementations. Our analysis provides insights into different design choices and uncovers potential bugs. Concretely, we have found critical bugs in multiple QUIC implementations, which have been acknowledged by the developers.

[1]  Nikhil Swamy,et al.  Everest: Towards a Verified, Drop-in Replacement of HTTPS , 2017, SNAPL.

[2]  W. M. McKeeman,et al.  Differential Testing for Software , 1998, Digit. Tech. J..

[3]  Joeri de Ruiter,et al.  Analysis of DTLS Implementations Using Protocol State Fuzzing , 2020, USENIX Security Symposium.

[4]  Joeri de Ruiter,et al.  Formal Models of Bank Cards for Free , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[5]  Klaus Wehrle,et al.  Interoperability-Guided Testing of QUIC Implementations using Symbolic Execution , 2018, EPIQ@CoNEXT.

[6]  Joeri de Ruiter.  A Tale of the OpenSSL State Machine: A Large-Scale Black-Box Analysis , 2016, NordSec.

[7]  Testing QUIC with packetdrill , 2020, EPIQ@SIGCOMM.

[8]  Mehryar Mohri,et al.  Learning Weighted Automata , 2015, CAI.

[9]  Harry B. Hunt,et al.  On Equivalence and Containment Problems for Formal Languages , 1977, JACM.

[10]  Frits W. Vaandrager,et al.  Combining Model Learning and Model Checking to Analyze TCP Implementations , 2016, CAV.

[11]  Angelos D. Keromytis,et al.  SFADiff: Automated Evasion Attacks and Fingerprinting Using Black-box Differential Automata Learning , 2016, CCS.

[12]  Yi Zhou,et al.  A Security Model and Fully Verified Implementation for the IETF QUIC Record Layer , 2020, 2021 IEEE Symposium on Security and Privacy (SP).

[13]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[14]  Frits W. Vaandrager,et al.  Model learning , 2017, Commun. ACM.

[15]  Olivier Bonaventure,et al.  Observing the Evolution of QUIC Implementations , 2018, EPIQ@CoNEXT.

[16]  Martin Thomson,et al.  QUIC: A UDP-Based Multiplexed and Secure Transport , 2020, RFC.

[17]  Frits W. Vaandrager,et al.  Inference and Abstraction of the Biometric Passport , 2010, ISoLA.

[18]  Kenneth L. McMillan,et al.  Formal specification and testing of QUIC , 2019, SIGCOMM.

[19]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[20]  Bernhard Steffen,et al.  The Open-Source LearnLib - A Framework for Active Automata Learning , 2015, CAV.

[21]  Jon Postel,et al.  DOD standard transmission control protocol , 1980, CCRV.

[22]  Joeri de Ruiter,et al.  Protocol State Fuzzing of TLS Implementations , 2015, USENIX Security Symposium.