Cali: Compiler-Assisted Library Isolation

Software libraries can freely access the program's entire address space, and also inherit its system-level privileges. This lack of separation regularly leads to security-critical incidents once libraries contain vulnerabilities or turn rogue. We present Cali, a compiler-assisted library isolation system that fully automatically shields a program from a given library. Cali is fully compatible with mainline Linux and does not require supervisor privileges to execute. We compartmentalize libraries into their own process with well-defined security policies. To preserve the functionality of the interactions between program and library, Cali uses a Program Dependence Graph to track data flow between the program and the library during link time. We evaluate our open-source prototype against three popular libraries: Ghostscript, OpenSSL, and SQLite. Cali successfully reduced the amount of memory that is shared between the program and library to 0.08% (ImageMagick) - 0.4% (Socat), while retaining an acceptable program performance.

[1]  Michalis Polychronakis,et al.  Temporal System Call Specialization for Attack Surface Reduction , 2020, USENIX Security Symposium.

[2]  Ben Niu,et al.  Enforcing user-space privilege separation with declarative architectures , 2012, STC '12.

[3]  Mark Handley,et al.  Wedge: Splitting Applications into Reduced-Privilege Compartments , 2008, NSDI.

[4]  Zhenkai Liang,et al.  Identifying Arbitrary Memory Access Vulnerabilities in Privilege-Separated Software , 2015, ESORICS.

[5]  Yutao Liu,et al.  Thwarting Memory Disclosure with Efficient Hypervisor-enforced Intra-domain Isolation , 2015, CCS.

[6]  Peter Druschel,et al.  ERIM: Secure, Efficient In-process Isolation with Protection Keys (MPK) , 2019, USENIX Security Symposium.

[7]  Thomas Schuster,et al.  Automated Detection, Exploitation, and Elimination of Double-Fetch Bugs using Modern CPU Features , 2017, AsiaCCS.

[8]  Peter G. Neumann,et al.  Clean Application Compartmentalization with SOAAP , 2015, CCS.

[9]  Niels Provos,et al.  Preventing Privilege Escalation , 2003, USENIX Security Symposium.

[10]  Douglas Kilpatrick,et al.  Privman: A Library for Partitioning Applications , 2003, USENIX Annual Technical Conference, FREENIX Track.

[11]  David Brumley,et al.  Privtrans: Automatically Partitioning Programs for Privilege Separation , 2004, USENIX Security Symposium.

[12]  Stefan Mangard,et al.  Donky: Domain Keys - Efficient In-Process Isolation for RISC-V and x86 , 2020, USENIX Security Symposium.

[13]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[14]  Long Lu,et al.  Shreds: Fine-Grained Execution Units with Private Memory , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[15]  Robert B. Miller,et al.  Response time in man-computer conversational transactions , 1899, AFIPS Fall Joint Computing Conference.

[16]  Pengfei Wang,et al.  How Double-Fetch Situations turn into Double-Fetch Vulnerabilities: A Study of Double Fetches in the Linux Kernel , 2017, USENIX Security Symposium.

[17]  Patrick Th. Eugster,et al.  Enforcing Least Privilege Memory Views for Multithreaded Applications , 2016, CCS.

[18]  Jun Sun,et al.  Automatically partition software into least privilege components using dynamic data dependency analysis , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[19]  Adrian Perrig,et al.  TrustVisor: Efficient TCB Reduction and Attestation , 2010, 2010 IEEE Symposium on Security and Privacy.

[20]  Jun Wang,et al.  Between Mutual Trust and Mutual Distrust: Practical Fine-grained Privilege Separation in Multithreaded Applications , 2013, USENIX Annual Technical Conference.

[21]  Joe D. Warren,et al.  The program dependence graph and its use in optimization , 1987, TOPL.

[22]  Jing Wang,et al.  Protecting Private Keys against Memory Disclosure Attacks Using Hardware Transactional Memory , 2015, 2015 IEEE Symposium on Security and Privacy.

[23]  Sorin Lerner,et al.  Retrofitting Fine Grain Isolation in the Firefox Renderer (Extended Version) , 2020, USENIX Security Symposium.

[24]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[25]  Hai Jin,et al.  Libsec: A Hardware Virtualization-Based Isolation for Shared Library , 2017, 2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[26]  Stephen McCamant,et al.  Program-mandering: Quantitative Privilege Separation , 2019, CCS.

[27]  Rodrigo Fonseca,et al.  sysfilter: Automated System Call Filtering for Commodity Software , 2020, RAID.

[28]  Azzedine Benameur,et al.  Confine: Automated System Call Policy Generation for Container Attack Surface Reduction , 2020, RAID.

[29]  Kevin W. Hamlen,et al.  Object Flow Integrity , 2017, CCS.

[30]  Dawn Xiaodong Song,et al.  SoK: Eternal War in Memory , 2013, 2013 IEEE Symposium on Security and Privacy.

[31]  Yu-Ping Wang,et al.  IVT: an efficient method for sharing subtype polymorphic objects , 2019, Proc. ACM Program. Lang..

[32]  Trent Jaeger,et al.  PtrSplit: Supporting General Pointers in Automatic Program Partitioning , 2017, CCS.

[33]  Brent Byunghoon Kang,et al.  Lord of the x86 Rings: A Portable User Mode Privilege Separation Architecture on x86 , 2018, CCS.

[34]  Frank Piessens,et al.  Salus: Kernel Support for Secure Process Compartments , 2015, EAI Endorsed Trans. Security Safety.

[35]  David Lo,et al.  Mining Sandboxes for Linux Containers , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[36]  Peter Druschel,et al.  Light-Weight Contexts: An OS Abstraction for Safety and Performance , 2016, OSDI.

[37]  Zhenkai Liang,et al.  Codejail: Application-Transparent Isolation of Libraries with Tight Program Interactions , 2012, ESORICS.