Characterizing kernel malware behavior with kernel data access patterns

Characterizing malware behavior using its control flow faces several challenges, such as obfuscations in static analysis and the behavior variations in dynamic analysis. This paper introduces a new approach to characterizing kernel malware's behavior by using kernel data access patterns unique to the malware. The approach neither uses malware's control flow consisting of temporal ordering of malware code execution, nor the code-specific information about the malware. Thus, the malware signature based on such data access patterns is resilient in matching malware variants. To evaluate the effectiveness of this approach, we first generated the signatures of three classic rootkits using their data access patterns, and then matched them with a group of kernel execution instances which are benign or compromised by 16 kernel rootkits. The malware signatures did not trigger any false positives in benign kernel runs; however, kernel runs compromised by 16 rootkits were detected due to the data access patterns shared with the compared signature(s). We further observed similar data access patterns in the signatures of the tested rootkits and exposed popular rootkit attack operations by ranking common data behavior across rootkits. Our experiments show that our approach is effective not only to detect the malware whose signature is available, but also to determine its variants which share kernel data access patterns.

[1]  Somesh Jha,et al.  Static Analysis of Executables to Detect Malicious Patterns , 2003, USENIX Security Symposium.

[2]  Xuxian Jiang,et al.  Mapping kernel objects to enable systematic integrity checking , 2009, CCS.

[3]  Samuel T. King,et al.  Digging for Data Structures , 2008, OSDI.

[4]  Somesh Jha,et al.  Mining specifications of malicious behavior , 2008, ISEC '08.

[5]  Christopher Krügel,et al.  Scalable, Behavior-Based Malware Clustering , 2009, NDSS.

[6]  Arati Baliga,et al.  Automatic Inference and Enforcement of Kernel Data Structure Invariants , 2008, 2008 Annual Computer Security Applications Conference (ACSAC).

[7]  Xuxian Jiang,et al.  SigGraph: Brute Force Scanning of Kernel Data Structure Instances Using Graph-based Signatures , 2011, NDSS.

[8]  Clark Thomborson,et al.  Manufacturing cheap, resilient, and stealthy opaque constructs , 1998, POPL '98.

[9]  Adrian Perrig,et al.  SecVisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity OSes , 2007, SOSP.

[10]  Sergey Bratus,et al.  SegSlice: Towards a New Class of Secure Programming Primitives for Trustworthy Platforms , 2010, TRUST.

[11]  Xuxian Jiang,et al.  Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing , 2008, RAID.

[12]  Christopher Krügel,et al.  Efficient Detection of Split Personalities in Malware , 2010, NDSS.

[13]  William A. Arbaugh,et al.  Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor , 2004, USENIX Security Symposium.

[14]  Raheem A. Beyah,et al.  Toward Revealing Kernel Malware Behavior in Virtual Execution Environments , 2009, RAID.

[15]  Xuxian Jiang,et al.  Kernel Malware Analysis with Un-tampered and Temporal Views of Dynamic Kernel Memory , 2010, RAID.

[16]  Christopher Krügel,et al.  Effective and Efficient Malware Detection at the End Host , 2009, USENIX Security Symposium.

[17]  Abhinav Srivastava,et al.  Robust signatures for kernel data structures , 2009, CCS.

[18]  William A. Arbaugh,et al.  An Architecture for Specification-Based Detection of Semantic Integrity Violations in Kernel Dynamic Data , 2006, USENIX Security Symposium.

[19]  Christopher Krügel,et al.  Detecting kernel-level rootkits through binary analysis , 2004, 20th Annual Computer Security Applications Conference.

[20]  Fabrice Bellard,et al.  QEMU, a Fast and Portable Dynamic Translator , 2005, USENIX Annual Technical Conference, FREENIX Track.

[21]  Jack W. Davidson,et al.  Protection of software-based survivability mechanisms , 2001, 2001 International Conference on Dependable Systems and Networks.

[22]  Xuxian Jiang,et al.  Multi-aspect profiling of kernel rootkit behavior , 2009, EuroSys '09.

[23]  Jonathon T. Giffin,et al.  Impeding Malware Analysis Using Conditional Code Obfuscation , 2008, NDSS.

[24]  Felix C. Freiling,et al.  Return-Oriented Rootkits: Bypassing Kernel Code Integrity Protection Mechanisms , 2009, USENIX Security Symposium.

[25]  Michael W. Hicks,et al.  Automated detection of persistent kernel control-flow attacks , 2007, CCS '07.