Rootkit detection from outside the Matrix

The main purpose of this article is to present a secure engine which is specifically designed for a security analyst when studying rootkits and all kinds of programs which interact at a deep level with the operating system, including Anti-Virus, Personal Firewall and HIPS programs. State-of-the-Art algorithms for rootkit detection are pre- sented in this paper. Forensic techniques to monitor the system’s critical components and advanced heuristics are also used. This survey is based on a proof-of-concept human analysis framework which puts forward a reliable system for automatically gaining information about a rootkit and its interaction with the OS executive, but focuses on human decision as a detection process without the same limitations or constraints as product-oriented anti-rootkit programs. We use the new point of view provided by this framework to take a fresh look at heuristics and forensics which are currently used by rootkit detectors.

[1]  Eric Filiol,et al.  A statistical model for undecidable viral detection , 2007, Journal in Computer Virology.

[2]  Peter Ferrie.  Attacks on Virtual Machine Emulators , 2007 .

[3]  Sébastien Josse.  Secure and advanced unpacking using computer emulation , 2007, Journal in Computer Virology.

[4]  U. Bayer,et al.  TTAnalyze: A Tool for Analyzing Malware , 2006 .

[5]  Éric Filiol.  Computer Viruses: from Theory to Applications , 2005 .

[6]  Eric Filiol,et al.  Malware Pattern Scanning Schemes Secure Against Black-box Analysis , 2006, Journal in Computer Virology.

[7]  J. Heasman.  Implementing and Detecting a PCI Rootkit , 2006 .

[8]  Fabrice Bellard,et al.  QEMU, a Fast and Portable Dynamic Translator , 2005, USENIX ATC, FREENIX Track.

[9]  William A. Arbaugh,et al.  Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor , 2004, USENIX Security Symposium.

[10]  Mark Russinovich,et al.  Microsoft Windows Internals, Fourth Edition: Microsoft Windows Server(TM) 2003, Windows XP, and Windows 2000 (Pro-Developer) , 2004 .

[11]  Mark Russinovich,et al.  Inside Microsoft Windows 2000 , 2000 .

[12]  Peter Szor,et al.  The Art of Computer Virus Research and Defense , 2005 .

[13]  John Heasman.  Rootkits: Rootkit threats , 2006 .

[14]  Mingtian Zhou,et al.  Some Further Theoretical Results about Computer Viruses , 2004, Comput. J..

[15]  Joanna Rutkowska.  Detecting Windows Server Compromises with Patchfinder 2 , 2004 .

[16]  Greg Hoglund,et al.  Rootkits: Subverting the Windows Kernel , 2005 .