How to kill symbolic deobfuscation for free (or: unleashing the potential of path-oriented protections)

Code obfuscation is a major tool for protecting software intellectual property from attacks such as reverse engineering or code tampering. Yet, recently proposed (automated) attacks based on Dynamic Symbolic Execution (DSE) shows very promising results, hence threatening software integrity. Current defenses are not fully satisfactory, being either not efficient against symbolic reasoning, or affecting runtime performance too much, or being too easy to spot. We present and study a new class of anti-DSE protections coined as path-oriented protections targeting the weakest spot of DSE, namely path exploration. We propose a lightweight, efficient, resistant and analytically proved class of obfuscation algorithms designed to hinder DSE-based attacks. Extensive evaluation demonstrates that these approaches critically counter symbolic deobfuscation while yielding only a very slight overhead.

[1]  Yin Liu,et al.  Static Information Flow Analysis with Handling of Implicit Flows and a Study on Effects of Implicit Flows vs Explicit Flows , 2010, 2010 14th European Conference on Software Maintenance and Reengineering.

[2]  Zhenkai Liang,et al.  Automatically Identifying Trigger-based Behavior in Malware , 2008, Botnet Detection.

[3]  Armin Biere,et al.  Bounded model checking , 2003, Adv. Comput..

[4]  Roberto Bruni,et al.  Code Obfuscation Against Abstract Model Checking Attacks , 2018, VMCAI.

[5]  David Brumley,et al.  Enhancing symbolic execution with veritesting , 2014, ICSE.

[6]  Jack W. Davidson,et al.  Software Tamper Resistance: Obstructing Static Analysis of Programs , 2000 .

[7]  David Brumley,et al.  Unleashing Mayhem on Binary Code , 2012, 2012 IEEE Symposium on Security and Privacy.

[8]  Jack W. Davidson,et al.  Protection of software-based survivability mechanisms , 2001, 2001 International Conference on Dependable Systems and Networks.

[9]  Saumya K. Debray,et al.  Reverse Engineering Self-Modifying Code: Unpacker Extraction , 2010, 2010 17th Working Conference on Reverse Engineering.

[10]  Saumya Debray,et al.  A Generic Approach to Automatic Deobfuscation of Executable Code , 2015, 2015 IEEE Symposium on Security and Privacy.

[11]  Christian S. Collberg,et al.  Probabilistic Obfuscation Through Covert Channels , 2018, 2018 IEEE European Symposium on Security and Privacy (EuroS&P).

[12]  Saumya Debray,et al.  Symbolic Execution of Obfuscated Code , 2015, CCS.

[13]  Stefan Katzenbeisser,et al.  Protecting Software through Obfuscation , 2016, ACM Comput. Surv..

[14]  Christian S. Collberg,et al.  Distributed application tamper detection via continuous software updates , 2012, ACSAC '12.

[15]  Jean-Yves Marion,et al.  Backward-Bounded DSE: Targeting Infeasibility Questions on Obfuscated Codes , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[16]  Hovav Shacham,et al.  The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86) , 2007, CCS '07.

[17]  Marco Torchiano,et al.  Understanding the behaviour of hackers while performing attack tasks in a professional setting and in a public challenge , 2018, Empirical Software Engineering.

[18]  Guillaume Bonfante,et al.  CoDisasm: Medium Scale Concatic Disassembly of Self-Modifying Binaries with Overlapping Instructions , 2015, CCS.

[19]  Thomas W. Reps,et al.  An improved algorithm for slicing machine code , 2016, OOPSLA.

[20]  Jean-Yves Marion,et al.  Specification of concretization and symbolization policies in symbolic execution , 2016, ISSTA.

[21]  Christopher Krügel,et al.  SOK: (State of) The Art of War: Offensive Techniques in Binary Analysis , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[22]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[23]  Patrice Godefroid,et al.  SAGE: Whitebox Fuzzing for Security Testing , 2012, ACM Queue.

[24]  Louis Goubin,et al.  Defeating MBA-based Obfuscation , 2016, SPRO@CCS.

[25]  Thomas A. Henzinger,et al.  Lazy abstraction , 2002, POPL '02.

[26]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[27]  Trent Jaeger,et al.  Implicit Flows: Can't Live with 'Em, Can't Live without 'Em , 2008, ICISS.

[28]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[29]  Heng Yin,et al.  Renovo: a hidden code extractor for packed executables , 2007, WORM '07.

[30]  Alexander Pretschner,et al.  Predicting the Resilience of Obfuscated Code Against Symbolic Execution Attacks via Machine Learning , 2017, USENIX Security Symposium.

[31]  Debin Gao,et al.  Linear Obfuscation to Combat Symbolic Execution , 2011, ESORICS.

[32]  Sean Heelan,et al.  SMT Solvers in Software Security , 2012, WOOT.

[33]  Cesare Tinelli,et al.  Satisfiability Modulo Theories , 2021, Handbook of Satisfiability.

[34]  Christian S. Collberg,et al.  Surreptitious Software - Obfuscation, Watermarking, and Tamperproofing for Software Protection , 2009, Addison-Wesley Software Security Series.

[35]  Jean-Yves Marion,et al.  BINSEC/SE: A Dynamic Symbolic Execution Toolkit for Binary-Level Analysis , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[36]  Krzysztof Czarnecki,et al.  Adaptive Restart and CEGAR-Based Solver for Inverting Cryptographic Hash Functions , 2016, VSTTE.

[37]  Jonathan Salwan,et al.  Symbolic Deobfuscation: From Virtualized Code Back to the Original , 2018, DIMVA.

[38]  Jonathon T. Giffin,et al.  Impeding Malware Analysis Using Conditional Code Obfuscation , 2008, NDSS.

[39]  Thomas W. Reps,et al.  WYSINWYX: What you see is not what you eXecute , 2005, TOPL.

[40]  Armin Biere,et al.  Boolector: An Efficient SMT Solver for Bit-Vectors and Arrays , 2009, TACAS.

[41]  David Brumley,et al.  All You Ever Wanted to Know about Dynamic Taint Analysis and Forward Symbolic Execution (but Might Have Been Afraid to Ask) , 2010, 2010 IEEE Symposium on Security and Privacy.

[42]  Yuan Xiang Gu,et al.  Information Hiding in Software with Mixed Boolean-Arithmetic Transforms , 2007, WISA.

[43]  Alexander Pretschner,et al.  Code obfuscation against symbolic execution attacks , 2016, ACSAC.

[44]  Amit Sahai,et al.  On the (im)possibility of obfuscating programs , 2001, JACM.

[45]  Johannes Kinder.  Towards Static Analysis of Virtualization-Obfuscated Binaries , 2012, 2012 19th Working Conference on Reverse Engineering.

[46]  Cristian Cadar,et al.  Targeted program transformations for symbolic execution , 2015, ESEC/SIGSOFT FSE.

[47]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[48]  Christian S. Collberg,et al.  A Taxonomy of Obfuscating Transformations , 1997 .

[49]  Axel Legay,et al.  Effectiveness of synthesis in concolic deobfuscation , 2017, Comput. Secur..

[50]  Kevin Coogan,et al.  Deobfuscation of virtualization-obfuscated software: a semantics-based approach , 2011, CCS '11.