CAB-Fuzz: Practical Concolic Testing Techniques for COTS Operating Systems

Discovering the security vulnerabilities of commercial off-the-shelf (COTS) operating systems (OSes) is challenging because they not only are huge and complex, but also lack detailed debug information. Concolic testing, which generates all feasible inputs of a program by using symbolic execution and tests the program with the generated inputs, is one of the most promising approaches to solve this problem. Unfortunately, the state-of-the-art concolic testing tools do not scale well for testing COTS OSes because of state explosion. Indeed, they often fail to find a single bug (or crash) in COTS OSes despite their long execution time. In this paper, we propose CAB-FUZZ (Context-Aware and Boundary-focused), a practical concolic testing tool to quickly explore interesting paths that are highly likely triggering real bugs without debug information. First, CAB-FUZZ prioritizes the boundary states of arrays and loops, inspired by the fact that many vulnerabilities originate from a lack of proper boundary checks. Second, CAB-FUZZ exploits real programs interacting with COTS OSes to construct proper contexts to explore deep and complex kernel states without debug information. We applied CAB-FUZZ to Windows 7 and Windows Server 2008 and found 21 undisclosed unique crashes, including two local privilege escalation vulnerabilities (CVE- 2015-6098 and CVE-2016-0040) and one information disclosure vulnerability in a cryptography driver (CVE- 2016-7219). CAB-FUZZ found vulnerabilities that are non-trivial to discover; five vulnerabilities have existed for 14 years, and we could trigger them even in the initial version of Windows XP (August 2001).

[1]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[2]  Jooyong Yi,et al.  Bogor/Kiasan: A k-bounded Symbolic Execution for Checking Strong Heap Properties of Open Systems , 2006, 21st IEEE/ACM International Conference on Automated Software Engineering (ASE'06).

[3]  Herbert Bos,et al.  Howard: A Dynamic Excavator for Reverse Engineering Data Structures , 2011, NDSS.

[4]  Nikolai Tillmann,et al.  Guided test generation for coverage criteria , 2010, 2010 IEEE International Conference on Software Maintenance.

[5]  Gordon Fraser,et al.  Augmented dynamic symbolic execution , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[6]  Alexander Aiken,et al.  Scalable error detection using boolean satisfiability , 2005, POPL '05.

[7]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[8]  George Candea,et al.  S2E: a platform for in-vivo multi-path analysis of software systems , 2011, ASPLOS XVI.

[9]  Eric Lahtinen,et al.  Targeted Automatic Integer Overflow Discovery Using Goal-Directed Conditional Branch Enforcement , 2015, ASPLOS.

[10]  Sarfraz Khurshid,et al.  Generalized Symbolic Execution for Model Checking and Testing , 2003, TACAS.

[11]  Dawson R. Engler,et al.  EXE: Automatically Generating Inputs of Death , 2008, TSEC.

[12]  George Candea,et al.  Testing Closed-Source Binary Device Drivers with DDT , 2010, USENIX Annual Technical Conference.

[13]  Milo M. K. Martin,et al.  SoftBound: highly compatible and complete spatial memory safety for c , 2009, PLDI '09.

[14]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[15]  Tao Wei,et al.  IntScope: Automatically Detecting Integer Overflow Vulnerability in X86 Binary Using Symbolic Execution , 2009, NDSS.

[16]  Dawson R. Engler,et al.  Under-constrained execution: making automatic code destruction easy and scalable , 2007, ISSTA '07.

[17]  Patrice Godefroid,et al.  Automatic partial loop summarization in dynamic test generation , 2011, ISSTA '11.

[18]  Bruno Legeard,et al.  Boundary coverage criteria for test generation from formal models , 2004, 15th International Symposium on Software Reliability Engineering.

[19]  David Brumley,et al.  Unleashing Mayhem on Binary Code , 2012, 2012 IEEE Symposium on Security and Privacy.

[20]  Herbert Bos,et al.  Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations , 2013, USENIX Security Symposium.

[21]  Stephen McCamant,et al.  Statically-directed dynamic automated test generation , 2011, ISSTA '11.

[22]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[23]  Stephen McCamant,et al.  Path-exploration lifting: hi-fi tests for lo-fi emulators , 2012, ASPLOS XVII.

[24]  Christopher Krügel,et al.  Driller: Augmenting Fuzzing Through Selective Symbolic Execution , 2016, NDSS.

[25]  Xiaoyin Wang,et al.  Experience report: how is dynamic symbolic execution different from manual testing? a study on KLEE , 2015, ISSTA.

[26]  Dawson R. Engler,et al.  Under-Constrained Symbolic Execution: Correctness Checking for Real Code , 2015, USENIX Annual Technical Conference.

[27]  Patrice Godefroid,et al.  Billions and billions of constraints: Whitebox fuzz testing in production , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[28]  Stephen McCamant,et al.  Loop-extended symbolic execution on binary programs , 2009, ISSTA.

[29]  Junfeng Yang,et al.  Automatically generating malicious disks using symbolic execution , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).

[30]  David Brumley,et al.  Enhancing symbolic execution with veritesting , 2014, ICSE.

[31]  William K. Robertson,et al.  LAVA: Large-Scale Automated Vulnerability Addition , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[32]  George Candea,et al.  Code-pointer integrity , 2014, OSDI.

[33]  David A. Wagner,et al.  Dynamic Test Generation to Find Integer Bugs in x86 Binary Linux Programs , 2009, USENIX Security Symposium.

[34]  Christopher Krügel,et al.  Firmalice - Automatic Detection of Authentication Bypass Vulnerabilities in Binary Firmware , 2015, NDSS.

[35]  Asim Kadav,et al.  SymDrive: Testing Drivers without Devices , 2012, OSDI.

[36]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[37]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[38]  Cristian Cadar,et al.  make test-zesti: A symbolic execution solution for improving regression testing , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[39]  Gordon Fraser,et al.  Generating Test Suites with Augmented Dynamic Symbolic Execution , 2013, TAP@STAF.

[40]  Dawson R. Engler,et al.  Redundant State Detection for Dynamic Symbolic Execution , 2013, USENIX Annual Technical Conference.