Hooktracer: Automatic Detection and Analysis of Keystroke Loggers Using Memory Forensics

Abstract Advances in malware development have led to the widespread use of attacker toolkits that do not leave any trace in the local filesystem. This negatively impacts traditional investigative procedures that rely on filesystem analysis to reconstruct attacker activities. As a solution, memory forensics has replaced filesystem analysis in these scenarios. Unfortunately, existing memory forensics tools leave many capabilities inaccessible to all but the most experienced investigators, who are well versed in operating systems internals and reverse engineering. The goal of the research described in this paper is to make investigation of one of the greatest threats that organizations face, userland keyloggers, less error-prone and less dependent on manual reverse engineering. To accomplish this, we have added significant new capabilities to HookTracer, which is an engine capable of emulating code discovered in a physical memory captures and recording all actions taken by the emulated code. Based on this work, we present new memory forensics capabilities, embodied in a new Volatility plugin, hooktracer_messagehooks, that uses Hooktracer to automatically decide whether a hook in memory is associated with a malicious keylogger or benign software. We also include a detailed case study that illustrates our technique’s ability to successfully analyze very sophisticated keyloggers, such as Turla.