Designing an efficient security framework for detecting intrusions in virtual network of cloud computing

Abstract Cloud computing has grown for various IT capabilities such as IoTs, Mobile Computing, Smart IT, etc. However, due to the dynamic and distributed nature of cloud and vulnerabilities existing in the current implementations of virtualization, several security threats and attacks have been reported. To address these issues, there is a need of extending traditional security solutions like firewall, intrusion detection/prevention systems which can cope up with high-speed network traffic and dynamic network configuration in the cloud. In addition, identifying feasible network traffic features is a major challenge for an accurate detection of the attacks. In this paper, we propose a hypervisor level distributed network security (HLDNS) framework which is deployed on each processing server of cloud computing. At each server, it monitors the underlying virtual machines (VMs) related network traffic to/from the virtual network, internal network and external network for intrusion detection. We have extended a binary bat algorithm (BBA) with two new fitness functions for deriving the feasible features from cloud network traffic. The derived features are applied to the Random Forest classifier for detecting the intrusions in cloud network traffic and intrusion alerts are generated. The intrusion alerts from different servers are correlated to identify the distributed attack and to generate new attack signature. For the performance and feasibility analysis, the proposed security framework is tested on the cloud network testbed at NIT Goa and using recent UNSW-NB15 and CICIDS-2017 intrusion datasets. We have performed a comparative analysis of the proposed security framework in terms of fulfilling the cloud network security needs.