KVMInspector: KVM Based introspection approach to detect malware in cloud environment

Abstract Cloud security is of paramount importance in this new era of computing. There are various security challenges in which researchers are still working on. Malware detection is one such domain. However, most of the work, mainly focuses on In-VM security techniques which can be evaded easily by malicious users. In this paper, we have proposed a dynamic analysis based introspection approach, called KVMInspector to detect malware in KVM-based cloud environment. The KVMInspector is deployed both inside and outside the VM at the KVM-layer and hence is more robust to attacks. LibVMI and Nitro libraries are used to extract the low-level details of a running virtual machine by viewing its memory, trapping hardware events, and accessing the vCPU registers from KVM. The KVMInspector first performs the process verification at KVM layer to provide a basic security check. After then, a detailed behaviour analysis is carried out to learn the dynamic (run-time) behaviour of the monitored programs using machine learning techniques as an advanced security check. A preliminary analysis has been performed using University of New Maxico (UNM) & University of California dataset and results seem to be promising.