Temporal System Call Specialization for Attack Surface Reduction

Attack surface reduction through the removal of unnecessary application features and code is a promising technique for improving security without incurring any additional overhead. Recent software debloating techniques consider an application’s entire lifetime when extracting its code requirements, and reduce the attack surface accordingly. In this paper, we present temporal specialization, a novel approach for limiting the set of system calls available to a process depending on its phase of execution. Our approach is tailored to server applications, which exhibit distinct initialization and serving phases with different system call requirements. We present novel static analysis techniques for improving the precision of extracting the application’s call graph for each execution phase, which is then used to pinpoint the system calls used in each phase. We show that requirements change throughout the lifetime of servers, and many dangerous system calls (such as execve) can be disabled after the completion of the initialization phase. We have implemented a prototype of temporal specialization on top of the LLVM compiler, and evaluated its effectiveness with six popular server applications. Our results show that it disables 51% more security-critical system calls compared to existing library specialization approaches, while offering the additional benefit of neutralizing 13 more Linux kernel vulnerabilities that could lead to privilege escalation.

[1]  Azzedine Benameur,et al.  Confine: Automated System Call Policy Generation for Container Attack Surface Reduction , 2020, RAID.

[2]  Jingling Xue,et al.  SVF: interprocedural static value-flow analysis in LLVM , 2016, CC.

[3]  Justin Cappos,et al.  Lock-in-Pop: Securing Privileged Operating System Kernels by Keeping on the Beaten Path , 2017, USENIX Annual Technical Conference.

[4]  Matti A. Hiltunen,et al.  Authenticated system calls , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[5]  Angelos D. Keromytis,et al.  kR^X: Comprehensive Kernel Protection against Just-In-Time Code Reuse , 2017, EuroSys.

[6]  Rodrigo Fonseca,et al.  sysfilter: Automated System Call Filtering for Commodity Software , 2020, RAID.

[7]  Ahmed M. Azab,et al.  PeX: A Permission Check Analysis Framework for Linux Kernel , 2019, USENIX Security Symposium.

[8]  Wolfgang Schröder-Preikschat,et al.  Attack Surface Metrics and Automated Compile-Time OS Kernel Tailoring , 2013, NDSS.

[9]  Zhongshu Gu,et al.  FACE-CHANGE: Application-Driven Dynamic Kernel View Switching in a Virtual Machine , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[10]  Aravind Prakash,et al.  Bloat Factors and Binary Specialization , 2019 .

[11]  Lok-Kwong Yan,et al.  Debloating Software through Piece-Wise Compilation and Loading , 2018, USENIX Security Symposium.

[12]  Peng Liu,et al.  JRed: Program Customization and Bloatware Mitigation Based on Static Analysis , 2016, 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC).

[13]  Andre Pawlowski,et al.  Towards Automated Application-Specific Software Stacks , 2019, ESORICS.

[14]  Jorge A. Navas,et al.  A Context-Sensitive Memory Model for Verification of C/C++ Programs , 2017, SAS.

[15]  Michalis Polychronakis,et al.  Shredder: Breaking Exploits through API Specialization , 2018, ACSAC.

[16]  David Lo,et al.  Mining Sandboxes for Linux Containers , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[17]  Olivier Barais,et al.  Learning Very Large Configuration Spaces: What Matters for Linux Kernel Sizes , 2019 .

[18]  Aravind Prakash,et al.  A Multi-OS Cross-Layer Study of Bloating in User Programs, Kernel and Managed Execution Environments , 2017, FEAST@CCS.

[19]  R. Sekar,et al.  A practical mimicry attack against powerful system-call monitors , 2008, ASIACCS '08.

[20]  Chenxiong Qian,et al.  RAZOR: A Framework for Post-deployment Software Debloating , 2019, USENIX Security Symposium.

[21]  Lars Ole Andersen,et al.  Program Analysis and Specialization for the C Programming Language , 2005 .

[22]  Calton Pu,et al.  TOCTTOU vulnerabilities in UNIX-style file systems: an anatomical study , 2005, FAST'05.

[23]  Christopher Krügel,et al.  Automating Mimicry Attacks Using Static Binary Analysis , 2005, USENIX Security Symposium.

[24]  Tal Garfinkel,et al.  Ostia: A Delegating Architecture for Secure System Call Interposition , 2004, NDSS.

[25]  Michalis Polychronakis,et al.  Configuration-Driven Software Debloating , 2019, EuroSec@EuroSys.

[26]  Hashim Sharif,et al.  Trimmer: Application Specialization for Code Debloating , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[27]  Jingling Xue,et al.  Detecting Memory Leaks Statically with Full-Sparse Value-Flow Analysis , 2014, IEEE Transactions on Software Engineering.

[28]  Somesh Jha,et al.  Formalizing sensitivity in static analysis for intrusion detection , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[29]  Bjarne Steensgaard,et al.  Points-to analysis in almost linear time , 1996, POPL '96.

[30]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[31]  Angelos D. Keromytis,et al.  ret2dir: Rethinking Kernel Isolation , 2014, USENIX Security Symposium.

[32]  Mangala Gowri Nanda,et al.  Combining concern input with program analysis for bloat detection , 2013, OOPSLA.

[33]  Jorge A. Navas,et al.  Unification-based Pointer Analysis without Oversharing , 2019, 2019 Formal Methods in Computer Aided Design (FMCAD).

[34]  Peng Liu,et al.  Feature-Based Software Customization: Preliminary Analysis, Formalization, and Methods , 2016, 2016 IEEE 17th International Symposium on High Assurance Systems Engineering (HASE).

[35]  R. Sekar,et al.  User-Level Infrastructure for System Call Interposition: A Platform for Intrusion Detection and Confinement , 2000, NDSS.

[36]  Dongxi Liu,et al.  KASR: A Reliable and Practical Approach to Attack Surface Reduction of Commodity OS Kernels , 2018, RAID.

[37]  Fernando Magno Quintão Pereira,et al.  Wave Propagation and Deep Propagation for Pointer Analysis , 2009, 2009 International Symposium on Code Generation and Optimization.

[38]  David A. Wagner,et al.  Intrusion detection via static analysis , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[39]  Michael Hind,et al.  Pointer analysis: haven't we solved this problem yet? , 2001, PASTE '01.

[40]  Di Jin,et al.  Nibbler: debloating binary shared libraries , 2019, ACSAC.

[41]  Prithayan Barua,et al.  BlankIt library debloating: getting what you want instead of cutting what you don’t , 2020, PLDI.

[42]  Steven McCanne,et al.  The BSD Packet Filter: A New Architecture for User-level Packet Capture , 1993, USENIX Winter.

[43]  Guru Venkataramani,et al.  DamGate: Dynamic Adaptive Multi-feature Gating in Program Binaries , 2017, FEAST@CCS.

[44]  Vasileios P. Kemerlis,et al.  Protecting Commodity Operating Systems through Strong Kernel Isolation , 2015 .

[45]  Kevin W. Hamlen,et al.  Binary Control-Flow Trimming , 2019, CCS.

[46]  Xinyu Xing,et al.  Fine-Grained Library Customization , 2018, ArXiv.

[47]  Rui Ma,et al.  SPEAKER: Split-Phase Execution of Application Containers , 2017, DIMVA.

[48]  Pierre Laperdrix,et al.  Less is More: Quantifying the Security Benefits of Debloating Web Applications , 2019, USENIX Security Symposium.

[49]  Mayur Naik,et al.  Effective Program Debloating via Reinforcement Learning , 2018, CCS.

[50]  Angelos D. Keromytis,et al.  kGuard: Lightweight Kernel Protection against Return-to-User Attacks , 2012, USENIX Security Symposium.

[51]  Gerard J. Holzmann,et al.  Code Inflation , 2015, IEEE Softw..

[52]  Michalis Polychronakis,et al.  Saffire: Context-sensitive Function Specialization against Code Reuse Attacks , 2020, 2020 IEEE European Symposium on Security and Privacy (EuroS&P).