Targeted Greybox Fuzzing with Static Lookahead Analysis

Automatic test generation typically aims to generate inputs that explore new paths in the program under test in order to find bugs. Existing work has, therefore, focused on guiding the exploration toward program parts that are more likely to contain bugs by using an offline static analysis. In this paper, we introduce a novel technique for targeted greybox fuzzing using an online static analysis that guides the fuzzer toward a set of target locations, for instance, located in recently modified parts of the program. This is achieved by first semantically analyzing each program path that is explored by an input in the fuzzer's test suite. The results of this analysis are then used to control the fuzzer's specialized power schedule, which determines how often to fuzz inputs from the test suite. We implemented our technique by extending a state-of-the-art, industrial fuzzer for Ethereum smart contracts and evaluate its effectiveness on 27 real-world benchmarks. Using an online analysis is particularly suitable for the domain of smart contracts since it does not require any code instrumentation-adding instrumentation to contracts changes their semantics. Our experiments show that targeted fuzzing significantly outperforms standard greybox fuzzing for reaching 83% of the challenging target locations (up to 14x of median speed-up).

[1]  Siraj Raval,et al.  Decentralized Applications: Harnessing Bitcoin's Blockchain Technology , 2016 .

[2]  Edmund M. Clarke,et al.  Counterexample-guided abstraction refinement , 2003, 10th International Symposium on Temporal Representation and Reasoning, 2003 and Fourth International Conference on Temporal Logic. Proceedings..

[3]  Cristian Cadar,et al.  KATCH: high-coverage testing of software patches , 2013, ESEC/FSE 2013.

[4]  Prateek Saxena,et al.  Making Smart Contracts Smarter , 2016, IACR Cryptol. ePrint Arch..

[5]  A. Vargha,et al.  A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong , 2000 .

[6]  Ondrej Lhoták,et al.  In defense of soundiness , 2015, Commun. ACM.

[7]  Greg Nelson,et al.  Extended static checking for Java , 2002, PLDI '02.

[8]  Nikolai Tillmann,et al.  Demand-Driven Compositional Symbolic Execution , 2008, TACAS.

[9]  Sidney Amani,et al.  Towards verifying ethereum smart contract bytecode in Isabelle/HOL , 2018, CPP.

[10]  Sven Apel,et al.  Views on Internal and External Validity in Empirical Software Engineering , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[11]  Gary A. Kildall,et al.  A unified approach to global program optimization , 1973, POPL.

[12]  Dawson R. Engler,et al.  Execution Generated Test Cases: How to Make Systems Code Crash Itself , 2005, SPIN.

[13]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[14]  Ye Liu,et al.  ContractFuzzer: Fuzzing Smart Contracts for Vulnerability Detection , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[15]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[16]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[17]  Daniel Davis Wood,et al.  ETHEREUM: A SECURE DECENTRALISED GENERALISED TRANSACTION LEDGER , 2014 .

[18]  Aditya V. Thakur,et al.  The Yogi Project : Software Property Checking via Static Analysis and Testing , 2009 .

[19]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[20]  Vincent Gramoli,et al.  Vandal: A Scalable Security Analysis Framework for Smart Contracts , 2018, ArXiv.

[21]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[22]  Raveendra Kumar Medicherla,et al.  VeriFuzz: Program Aware Fuzzing - (Competition Contribution) , 2019, TACAS.

[23]  Alex Groce,et al.  Slither: A Static Analysis Framework for Smart Contracts , 2019, 2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB).

[24]  Melanie Swan,et al.  Blockchain: Blueprint for a New Economy , 2015 .

[25]  Martin C. Rinard,et al.  Taint-based directed whitebox fuzzing , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[26]  Ilya Sergey,et al.  A Concurrent Perspective on Smart Contracts , 2017, Financial Cryptography Workshops.

[27]  GodefroidPatrice Compositional dynamic test generation , 2007 .

[28]  Krishnendu Chatterjee,et al.  Quantitative Analysis of Smart Contracts , 2018, ESOP.

[29]  Christian Rossow,et al.  teEther: Gnawing at Ethereum to Automatically Exploit Smart Contracts , 2018, USENIX Security Symposium.

[30]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[31]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[32]  Petar Tsankov,et al.  Securify: Practical Security Analysis of Smart Contracts , 2018, CCS.

[33]  SuZhendong,et al.  Steering symbolic execution to less traveled paths , 2013 .

[34]  Chen Fu,et al.  CarFast: achieving higher statement coverage faster , 2012, SIGSOFT FSE.

[35]  Yang Liu,et al.  Steelix: program-state based binary fuzzing , 2017, ESEC/SIGSOFT FSE.

[36]  Isil Dillig,et al.  Formal Verification of Workflow Policies for Smart Contracts in Azure Blockchain , 2019, VSTTE.

[37]  Nikhil Swamy,et al.  Formal Verification of Smart Contracts: Short Paper , 2016, PLAS@CCS.

[38]  Heike Wehrheim,et al.  Just Test What You Cannot Verify! , 2015, FASE.

[39]  Satish Narayanasamy,et al.  Optimistic Hybrid Analysis: Accelerating Dynamic Analysis through Predicated Static Analysis , 2018, ASPLOS.

[40]  Abhik Roychoudhury,et al.  Coverage-Based Greybox Fuzzing as Markov Chain , 2016, IEEE Transactions on Software Engineering.

[41]  Manuel Fähndrich,et al.  Static Contract Checking with Abstract Interpretation , 2010, FoVeOOS.

[42]  Ákos Hajdu,et al.  solc-verify: A Modular Verifier for Solidity Smart Contracts , 2019, VSTTE.

[43]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[44]  George Candea,et al.  Efficient state merging in symbolic execution , 2012, Software Engineering.

[45]  Zhendong Su,et al.  Steering symbolic execution to less traveled paths , 2013, OOPSLA.

[46]  Prateek Saxena,et al.  Finding The Greedy, Prodigal, and Suicidal Contracts at Scale , 2018, ACSAC.

[47]  C. Csallner,et al.  Check 'n' crash: combining static checking and testing , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[48]  Bihuan Chen,et al.  Hawkeye: Towards a Desired Directed Grey-box Fuzzer , 2018, CCS.

[49]  Koushik Sen,et al.  FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[50]  Xiapu Luo,et al.  Under-optimized smart contracts devour your money , 2017, 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[51]  Koushik Sen,et al.  CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools , 2006, CAV.

[52]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[53]  Sriram K. Rajamani,et al.  Compositional may-must program analysis: unleashing the power of alternation , 2010, POPL '10.

[54]  Andrew Ruef,et al.  Evaluating Fuzz Testing , 2018, CCS.

[55]  CandeaGeorge,et al.  Efficient state merging in symbolic execution , 2012 .

[56]  Peter Müller,et al.  Guiding Dynamic Symbolic Execution toward Unverified Program Executions , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[57]  Nikolai Tillmann,et al.  DyTa: dynamic symbolic execution guided with static verification results , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[58]  Patrick Cousot,et al.  Systematic design of program analysis frameworks , 1979, POPL.

[59]  Alex Groce,et al.  Manticore: A User-Friendly Symbolic Execution Framework for Binaries and Smart Contracts , 2019, 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[60]  Chunming Wu,et al.  V-Fuzz: Vulnerability-Oriented Evolutionary Fuzzing , 2019, ArXiv.

[61]  Matthew B. Dwyer,et al.  Residual dynamic typestate analysis exploiting static analysis: results to reformulate and reduce the cost of dynamic analysis , 2007, ASE.

[62]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[63]  Abhik Roychoudhury,et al.  Directed Greybox Fuzzing , 2017, CCS.

[64]  Sukrit Kalra,et al.  ZEUS: Analyzing Safety of Smart Contracts , 2018, NDSS.

[65]  Massimo Bartoletti,et al.  A Survey of Attacks on Ethereum Smart Contracts (SoK) , 2017, POST.

[66]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[67]  Herbert Bos,et al.  Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations , 2013, USENIX Security Symposium.

[68]  Valentin Wüstholz,et al.  Harvey: a greybox fuzzer for smart contracts , 2019, ESEC/SIGSOFT FSE.

[69]  Valentin Wüstholz,et al.  Learning Inputs in Greybox Fuzzing , 2018, ArXiv.

[70]  David Brumley,et al.  Enhancing symbolic execution with veritesting , 2014, ICSE.

[71]  GodefroidPatrice,et al.  Compositional may-must program analysis , 2010 .

[72]  Michael Hicks,et al.  Directed Symbolic Execution , 2011, SAS.

[73]  Isil Dillig,et al.  Failure-directed program trimming , 2017, ESEC/SIGSOFT FSE.

[74]  Yu Jiang,et al.  SAFL: Increasing and Accelerating Testing Coverage with Symbolic Execution and Guided Fuzzing , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion).

[75]  Yannis Smaragdakis,et al.  MadMax: surviving out-of-gas conditions in Ethereum smart contracts , 2018, Proc. ACM Program. Lang..

[76]  Ittai Abraham,et al.  Online detection of effectively callback free objects with applications to smart contracts , 2017, Proc. ACM Program. Lang..

[77]  Hiroyuki Sato,et al.  GRT: Program-Analysis-Guided Random Testing (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).