Lock-in-Pop: Securing Privileged Operating System Kernels by Keeping on the Beaten Path

Virtual machines (VMs) that try to isolate untrusted code are widely used in practice. However, it is often possible to trigger zero-day flaws in the host Operating System (OS) from inside of such virtualized systems. In this paper, we propose a new security metric showing strong correlation between “popular paths” and kernel vulnerabilities. We verify that the OS kernel paths accessed by popular applications in everyday use contain significantly fewer security bugs than less-used paths. We then demonstrate that this observation is useful in practice by building a prototype system which locks an application into using only popular OS kernel paths. By doing so, we demonstrate that we can prevent the triggering of zero-day kernel bugs significantly better than three other competing approaches, and argue that this is a practical approach to secure system design.

[1]  Neha Narula,et al.  Native Client: A Sandbox for Portable, Untrusted x86 Native Code , 2009, IEEE Symposium on Security and Privacy.

[2]  Xiaoxin Chen,et al.  Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems , 2008, ASPLOS.

[3]  Christophe Calvès,et al.  Faults in linux: ten years later , 2011, ASPLOS XVI.

[4]  Reuben Olinsky,et al.  Composing OS extensions safely and efficiently with Bascule , 2013, EuroSys '13.

[5]  Galen C. Hunt,et al.  Shielding Applications from an Untrusted Cloud with Haven , 2014, OSDI.

[6]  David Wagner,et al.  Janus: an Approach for Confinement of Untrusted Applications , 1999 .

[7]  Stuart E. Schechter,et al.  Milk or Wine: Does Software Security Improve with Age? , 2006, USENIX Security Symposium.

[8]  Robert Wahbe,et al.  Efficient software-based fault isolation , 1994, SOSP '93.

[9]  Tal Garfinkel,et al.  Ostia: A Delegating Architecture for Secure System Call Interposition , 2004, NDSS.

[10]  Kang G. Shin,et al.  Using hypervisor to provide data secrecy for user applications on a per-page basis , 2008, VEE '08.

[11]  Donald E. Porter,et al.  Cooperation and security isolation of library OSes for multi-process applications , 2014, EuroSys '14.

[12]  Ivan Beschastnikh,et al.  Retaining sandbox containment despite bugs in privileged memory-safe code , 2010, CCS '10.

[13]  Ian Goldberg,et al.  A Secure Environment for Untrusted Helper Applications ( Confining the Wily Hacker ) , 1996 .

[14]  Alan Mayer,et al.  A probability model for analysing complexity metrics data , 1989, Softw. Eng. J..

[15]  Donald E. Porter,et al.  Rethinking the library OS from the top down , 2011, ASPLOS XVI.

[16]  Nathanael Paul,et al.  Comparing Java and .NET security: Lessons learned and missed , 2006, Comput. Secur..

[17]  Jennifer Rexford,et al.  NoHype: virtualized cloud infrastructure without the virtualization , 2010, ISCA.

[18]  Niraj K. Jha,et al.  Secure Virtual Machine Execution under an Untrusted Management OS , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[19]  Lee J. Bain,et al.  Experiment Size and Power Comparisons for Two‐Sample Poisson Tests , 1982 .