Behavioural detection with API call-grams to identify malicious PE files

Present day malware shows stealthy and dynamic capability to avail administrative rights and control the victim computer [10]. Malware writers depend on evasion techniques like code obfuscation, packing, compression, encryption or polymorphism to avoid detection by Anti-Virus (AV) scanners as AV primarily use signature based detection. According to the FireEye Threat report second half of 2011 [15], top 50 malware have generated 80% infections. Malware like Zues, Conficker, Koobface have become more stealthy by use of pay per install toolkits like Blackhole [15]. Pay per install toolkits make the samples dynamic in nature. This has led to exponential increase of unknown, zero-day malware [14]. To complement the signatured approach, a good behavioral scheme is imminent due to exponential increase in number of encoded malware samples. Behavioural analysis can detect unknown, encrypted, zero day malware, but these methods result in increased false alarm rate. We propose a behaviour model that represents abstraction of a binary by analyzing the Application Programming Interface (API) strings made by Windows Portable Executable (PE) [25] files. Our focus is based on extracting temporal snapshots of malware and benign executables known as API Call-grams, as API strings are primarily written for software development kits to generate sane code. Malcode writers misues the available functionality to keep the code compact and escape being detected by AV software.