Scalable Greybox Fuzzing for Effective Vulnerability Management

We describe a dynamic analysis technique for discovering vulnerabilities where we, first, analyse isolated components of a program for vulnerabilities with three modes – symbolic execution, fuzzing and a novel greybox fuzzing method. To determine the feasibility of vulnerabilities, we propose a compositional analysis method using targeted symbolic execution. Finally, we discuss an adaptable assessment method based on heuristics from bug-repository- and code-mining, to assist in bug triage.

[1]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[2]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[3]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[4]  Abhik Roychoudhury,et al.  Model-based whitebox fuzzing for program binaries , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[5]  Bin Zhang,et al.  Discover deeper bugs with dynamic symbolic execution and coverage-based fuzz testing , 2018, IET Softw..

[6]  Lin Teng,et al.  Binary-oriented hybrid fuzz testing , 2015, 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS).

[7]  Herbert Bos,et al.  VUzzer: Application-aware Evolutionary Fuzzing , 2017, NDSS.

[8]  Eugene H. Spafford,et al.  Software vulnerability analysis , 1998 .

[9]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[10]  Alexander Pretschner,et al.  Reviewing KLEE's Sonar-Search Strategy in Context of Greybox Fuzzing , 2018, ArXiv.

[11]  Daniel P. Siewiorek,et al.  Automated robustness testing of off-the-shelf software components , 1998, Digest of Papers. Twenty-Eighth Annual International Symposium on Fault-Tolerant Computing (Cat. No.98CB36224).

[12]  Sebastian-Emilian Banescu.  Characterizing the Strength of Software Obfuscation Against Automated Attacks , 2017 .

[13]  Derek Bruening,et al.  AddressSanitizer: A Fast Address Sanity Checker , 2012, USENIX Annual Technical Conference.

[14]  Claudia Eckert,et al.  Detection of Intrusions and Malware, and Vulnerability Assessment , 2016, Lecture Notes in Computer Science.

[15]  David Brumley,et al.  All You Ever Wanted to Know about Dynamic Taint Analysis and Forward Symbolic Execution (but Might Have Been Afraid to Ask) , 2010, 2010 IEEE Symposium on Security and Privacy.

[16]  Christopher Krügel,et al.  Driller: Augmenting Fuzzing Through Selective Symbolic Execution , 2016, NDSS.

[17]  Patrice Godefroid,et al.  SAGE: Whitebox Fuzzing for Security Testing , 2012, ACM Queue.

[18]  Gregory Tassey,et al.  Prepared for what , 2007 .

[19]  Zohar Manna,et al.  The calculus of computation - decision procedures with applications to verification , 2007 .

[20]  Qi Xi,et al.  Malware Behavior Analysis Technique Based on Approach to Sensitive Behavior Functions , 2012 .

[21]  Prathima Agrawal,et al.  Probabilistic Analysis of Random Test Generation Method for Irredundant Combinational Logic Networks , 1975, IEEE Transactions on Computers.

[22]  Muhammad Zubair Shafiq,et al.  A large scale exploratory analysis of software vulnerability life cycles , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[23]  David Brumley,et al.  Optimizing Seed Selection for Fuzzing , 2014, USENIX Security Symposium.

[24]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[25]  Jun He,et al.  Automatic software vulnerability detection based on guided deep fuzzing , 2014, 2014 IEEE 5th International Conference on Software Engineering and Service Science.

[26]  Hao Chen,et al.  Angora: Efficient Fuzzing by Principled Search , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[27]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[28]  Herbert Bos,et al.  Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations , 2013, USENIX Security Symposium.

[29]  Giuliano Antoniol,et al.  Threats on building models from CVS and Bugzilla repositories: the Mozilla case study , 2007, CASCON.

[30]  Emerson R. Murphy-Hill,et al.  The Design Space of Bug Fixes and How Developers Navigate It , 2015, IEEE Transactions on Software Engineering.

[31]  John Cocke,et al.  A program data flow analysis procedure , 1976, CACM.

[32]  David L. Dill,et al.  A Decision Procedure for Bit-Vectors and Arrays , 2007, CAV.

[33]  Paolo Falcarin,et al.  Guest Editors' Introduction: Software Protection , 2011, IEEE Software.

[34]  Patrice Godefroid,et al.  IC-Cut: A Compositional Search Strategy for Dynamic Test Generation , 2015, SPIN.

[35]  Daniel Kroening,et al.  Behavioral consistency of C and Verilog programs using bounded model checking , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[36]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[37]  Rupak Majumdar,et al.  Reducing Test Inputs Using Information Partitions , 2009, CAV.

[38]  George Candea,et al.  Efficient state merging in symbolic execution , 2012, Software Engineering.

[39]  Andrew Meneely,et al.  Do Bugs Foreshadow Vulnerabilities? A Study of the Chromium Project , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[40]  Gary McGraw,et al.  Exploiting Software: How to Break Code , 2004 .

[41]  Brian S. Pak,et al.  Hybrid Fuzz Testing: Discovering Software Bugs via Fuzzing and Symbolic Execution , 2012 .

[42]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[43]  George Candea,et al.  S2E: a platform for in-vivo multi-path analysis of software systems , 2011, ASPLOS XVI.

[44]  Michael S. Hsiao,et al.  Strategies for scalable symbolic execution-driven test generation for programs , 2011, Science China Information Sciences.

[45]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[46]  Javam C. Machado,et al.  The prediction of faulty classes using object-oriented design metrics , 2001, J. Syst. Softw..

[47]  Julian Petley.  Panic stations: surveillance in the UK , 2013 .

[48]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[49]  Rajiv D. Banker,et al.  Software complexity and maintenance costs , 1993, CACM.

[50]  Richard E. Fairley,et al.  Tutorial: Static Analysis and Dynamic Testing of Computer Software , 1978, Computer.

[51]  Koushik Sen,et al.  FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[52]  Pedram Amini,et al.  Fuzzing: Brute Force Vulnerability Discovery , 2007 .

[53]  Jens H. Weber,et al.  Hermes: A Targeted Fuzz Testing Framework , 2015, SoMeT.

[54]  N. Nagappan,et al.  Static analysis tools as early indicators of pre-release defect density , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[55]  Alexander Pretschner,et al.  A Taxonomy of Software Integrity Protection Techniques , 2018 .

[56]  Corina S. Pasareanu,et al.  Badger: complexity analysis with fuzzing and symbolic execution , 2018, ISSTA.

[57]  Standard Glossary of Software Engineering Terminology , 1990 .

[58]  Nikolai Tillmann,et al.  Demand-Driven Compositional Symbolic Execution , 2008, TACAS.

[59]  Cesare Tinelli,et al.  Satisfiability Modulo Theories , 2021, Handbook of Satisfiability.

[60]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[61]  Nikolai Tillmann,et al.  Fitness-guided path exploration in dynamic symbolic execution , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[62]  Alexander Pretschner,et al.  Improving function coverage with munch: a hybrid fuzzing and directed symbolic execution approach , 2017, SAC.

[63]  P. Samarati,et al.  Access control: principle and practice , 1994, IEEE Communications Magazine.

[64]  Tim Miller,et al.  Compositional Symbolic Execution Using Fine-Grained Summaries , 2015, 2015 24th Australasian Software Engineering Conference.

[65]  Christopher Krügel,et al.  Firmalice - Automatic Detection of Authentication Bypass Vulnerabilities in Binary Firmware , 2015, NDSS.

[66]  Corina S. Pasareanu,et al.  JPF-SE: A Symbolic Execution Extension to Java PathFinder , 2007, TACAS.

[67]  Noam Rinetzky,et al.  Chopped Symbolic Execution , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[68]  David J. Musliner,et al.  Using Concolic Testing to Refine Vulnerability Profiles in FUZZBUSTER , 2012, 2012 IEEE Sixth International Conference on Self-Adaptive and Self-Organizing Systems Workshops.

[69]  Michael Hicks,et al.  Directed Symbolic Execution , 2011, SAS.

[70]  Dawn Xiaodong Song,et al.  BLITZ: Compositional bounded model checking for real-world programs , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[71]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[72]  Meng Xu,et al.  QSYM : A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing , 2018, USENIX Security Symposium.

[73]  Matt Bishop,et al.  Computer Security: Art and Science , 2002 .

[74]  Jianmin Chen,et al.  Ewap: Using Symbolic Execution to Exploit Windows Applications , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[75]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[76]  Corina S. Pasareanu,et al.  A survey of new trends in symbolic execution for software testing and analysis , 2009, International Journal on Software Tools for Technology Transfer.

[77]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[78]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[79]  Koushik Sen,et al.  Heuristics for Scalable Dynamic Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[80]  Abhik Roychoudhury,et al.  Hercules: Reproducing Crashes in Real-World Application Binaries , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[81]  Guofei Gu,et al.  TaintScope: A Checksum-Aware Directed Fuzzing Tool for Automatic Software Vulnerability Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[82]  Abhik Roychoudhury,et al.  Directed Greybox Fuzzing , 2017, CCS.

[83]  Daniel Kroening,et al.  Decision Procedures , 2016, Texts in Theoretical Computer Science. An EATCS Series.

[84]  J. Scott-Railton,et al.  The Million Dollar Dissident: NSO Group’s iPhone Zero-Days used against a UAE Human Rights Defender , 2016 .

[85]  David Brumley,et al.  Enhancing symbolic execution with veritesting , 2014, ICSE.

[86]  Dorothy E. Denning,et al.  Cryptography and Data Security , 1982 .

[87]  David Brumley,et al.  Program-Adaptive Mutational Fuzzing , 2015, 2015 IEEE Symposium on Security and Privacy.

[88]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[89]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[90]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[91]  Dawson R. Engler,et al.  Execution Generated Test Cases: How to Make Systems Code Crash Itself , 2005, SPIN.

[92]  Shize Guo,et al.  A Directed Fuzzing Based on the Dynamic Symbolic Execution and Extended Program Behavior Model , 2012, 2012 Second International Conference on Instrumentation, Measurement, Computer, Communication and Control.

[93]  Kosta Serebryany,et al.  Continuous Fuzzing with libFuzzer and AddressSanitizer , 2016, 2016 IEEE Cybersecurity Development (SecDev).

[94]  Patrice Godefroid,et al.  Proving Memory Safety of the ANI Windows Image Parser Using Compositional Exhaustive Testing , 2015, VMCAI.

[95]  Victor R. Basili,et al.  The influence of organizational structure on software quality , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[96]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[97]  Gary McGraw,et al.  Software Security Testing , 2004, IEEE Secur. Priv..

[98]  Abhik Roychoudhury,et al.  Coverage-Based Greybox Fuzzing as Markov Chain , 2016, IEEE Transactions on Software Engineering.

[99]  Martin C. Rinard,et al.  Taint-based directed whitebox fuzzing , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[100]  Manu Sridharan,et al.  Alternate and Learn: Finding Witnesses without Looking All over , 2012, CAV.

[101]  Tao Xie,et al.  Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[102]  Kristian Beckers,et al.  An Exploratory Survey of Hybrid Testing Techniques Involving Symbolic Execution and Fuzzing , 2017, ArXiv.

[103]  Alexander Pretschner,et al.  MACKE: Compositional analysis of low-level vulnerabilities with symbolic execution , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[104]  Alexander Pretschner,et al.  Automatically assessing vulnerabilities discovered by compositional analysis , 2018, MASES@ASE.

[105]  David Hovemeyer,et al.  Using Static Analysis to Find Bugs , 2008, IEEE Software.

[106]  Claes Wohlin,et al.  Guidelines for snowballing in systematic literature studies and a replication in software engineering , 2014, EASE '14.

[107]  Pankaj Mudholkar,et al.  Software Testing , 2002, Computer.

[108]  Karen A. Scarfone,et al.  A Complete Guide to the Common Vulnerability Scoring System Version 2.0 | NIST , 2007 .

[109]  David Brumley,et al.  Unleashing Mayhem on Binary Code , 2012, 2012 IEEE Symposium on Security and Privacy.

[110]  Wang,et al.  An Instruction-level Symbolic Checksum System for Windows x86 Program , 2012 .

[111]  Yu Jiang,et al.  SAFL: Increasing and Accelerating Testing Coverage with Symbolic Execution and Guided Fuzzing , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion).

[112]  David L. Dill,et al.  A decision procedure for bit-vector arithmetic , 1998, Proceedings 1998 Design and Automation Conference. 35th DAC. (Cat. No.98CH36175).

[113]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[114]  Leon J. Osterweil,et al.  Data Flow Analysis in Software Reliability , 1976, CSUR.

[115]  Sergey Berezin,et al.  CVC Lite: A New Implementation of the Cooperating Validity Checker Category B , 2004, CAV.

[116]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[117]  Roland Groz,et al.  A Taint Based Approach for Smart Fuzzing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[118]  A. Pretschner.  Classical search strategies for test case generation with Constraint Logic Programming , 2001 .

[119]  Koushik Sen,et al.  MultiSE: multi-path symbolic execution using value summaries , 2015, ESEC/SIGSOFT FSE.