An Immediate System Call Sequence Based Approach for Detecting Malicious Program Executions in Cloud Environment

Cloud computing is a well-known architecture that provides Computing and data Storage services remotely over Internet on a pay per usage model which results in better utilization of resources with reduced cost for individuals to access it. As Cloud Computing is a shared facility and is accessed remotely, it is vulnerable to various attacks including hosts and network based attacks that require immediate attention. This paper focuses on attacks that are due to malicious Syscall executions from subverted programs, Rootkits, Worms and Trojans on Hosts in a Cloud Computing environment. The paper critically describes and discusses the present techniques for malicious System Call detection and proposes a new Immediate Syscall signature structure based technique to determine malicious program executions in Cloud. The proposed technique is efficient in terms of complexity involved and resources utilized by it, so as to justify its feasible deployment is low cost and platform independent in Cloud environment. The proposed technique has also been validated on all available UNM (University of New Mexico) datasets and with a 98% accuracy in program wide detection for detecting intrusive processes. The functional prototype is deployed on a private Cloud environment using open nebula and virtual box for analysis and results.