On the Detection of Kernel-Level Rootkits Using Hardware Performance Counters

Recent work has investigated the use of hardware performance counters (HPCs) for the detection of malware running on a system. These works gather traces of HPCs for a variety of applications (both malicious and non-malicious) and then apply machine learning to train a detector to distinguish between benign applications and malware. In this work, we provide a more comprehensive analysis of the applicability of using machine learning and HPCs for a specific subset of malware: kernel rootkits. We design five synthetic rootkits, each providing a single piece of rootkit functionality, and execute each while collecting HPC traces of its impact on a specific benchmark application. We then apply machine learning feature selection techniques in order to determine the most relevant HPCs for the detection of these rootkits. We identify 16 HPCs that are useful for the detection of hooking based roots, and also find that rootkits employing direct kernel object manipulation (DKOM) do not significantly impact HPCs. We then use these synthetic rootkit traces to train a detection system capable of detecting new rootkits it has not seen previously with an accuracy of over 99%. Our results indicate that HPCs have the potential to be an effective tool for rootkit detection, even against new rootkits not previously seen by the detector.

[1]  Xuxian Jiang,et al.  Countering kernel rootkits with lightweight hook protection , 2009, CCS.

[2]  Sally A. McKee,et al.  Can hardware performance counters be trusted? , 2008, 2008 IEEE International Symposium on Workload Characterization.

[3]  Nael B. Abu-Ghazaleh,et al.  Iso-X: A Flexible Architecture for Hardware-Managed Isolated Execution , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.

[4]  Rudolf Berrendorf,et al.  PCL - The Performance Counter Library: A Common Interface to Access Hardware Performance Counters on Microprocessors , 1998 .

[5]  Richard Luczak,et al.  The PAPI Cross-Platform Interface to Hardware Performance Counters , 2001 .

[6]  Ryan Riley A framework for prototyping and testing data-only rootkit attacks , 2013, Comput. Secur..

[7]  Sally A. McKee,et al.  Real time power estimation and thread scheduling via performance counters , 2009, CARN.

[8]  Bernd Eggers Rootkits Subverting The Windows Kernel , 2016 .

[9]  Nael B. Abu-Ghazaleh,et al.  Malware-aware processors: A framework for efficient online malware detection , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[10]  Ramesh Karri,et al.  NumChecker: Detecting kernel control-flow modifying rootkits by using Hardware Performance Counters , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).

[11]  Nicolas Le Scouarnec,et al.  Reverse Engineering Intel Last-Level Cache Complex Addressing Using Performance Counters , 2015, RAID.

[12]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[13]  Salvatore J. Stolfo,et al.  Unsupervised Anomaly-Based Malware Detection Using Hardware Features , 2014, RAID.

[14]  John M. May,et al.  MPX: Software for multiplexing hardware performance counters in multithreaded programs , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[15]  Ramesh Karri,et al.  Are hardware performance counters a cost effective way for integrity checking of programs , 2011, STC '11.

[16]  Jack J. Dongarra,et al.  A Portable Programming Interface for Performance Evaluation on Modern Processors , 2000, Int. J. High Perform. Comput. Appl..

[17]  Hovav Shacham,et al.  The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86) , 2007, CCS '07.

[18]  Guanhua Yan,et al.  Exploring Discriminatory Features for Automated Malware Classification , 2013, DIMVA.

[19]  Angelos D. Keromytis,et al.  Transparent ROP Exploit Mitigation Using Indirect Branch Tracing , 2013, USENIX Security Symposium.

[20]  Felix C. Freiling,et al.  Return-Oriented Rootkits: Bypassing Kernel Code Integrity Protection Mechanisms , 2009, USENIX Security Symposium.

[21]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[22]  Brinkley Sprunt,et al.  The Basics of Performance-Monitoring Hardware , 2002, IEEE Micro.

[23]  Salvatore J. Stolfo,et al.  On the feasibility of online malware detection with performance counters , 2013, ISCA.

[24]  References , 1971 .

[25]  Kevin Skadron,et al.  Using performance counters for runtime temperature sensing in high-performance processors , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[26]  Daniel Bilar,et al.  Opcodes as predictor for malware , 2007, Int. J. Electron. Secur. Digit. Forensics.

[27]  Yoseba K. Penya,et al.  Idea: Opcode-Sequence-Based Malware Detection , 2010, ESSoS.

[28]  Xuxian Jiang,et al.  Defeating Dynamic Data Kernel Rootkit Attacks via VMM-Based Guest-Transparent Monitoring , 2009, 2009 International Conference on Availability, Reliability and Security.