Protecting VNF services with smart online behavior anomaly detection method

Abstract Network Function Virtualization (NFV) is an emerging technology that allows network operators to deploy their Virtualized Network Functions (VNFs) on low-cost commodity servers in the cloud data center. The VNFs, such as virtual routers, firewalls etc., that typically control and transmit critical network packages, require strong security guarantees. However, detecting malicious or malfunctioning VNFs are challenging, as the behaviors of VNFs are dynamic and complex due to the changing network traffics in the cloud. In this paper, we propose a smart and efficient Hidden Markov Model based anomaly detection system (named vGuard) to protect online VNF services in the cloud. A general multivariate HMM model is proposed to profile the normal VNF behavior patterns. Using the VNF behavior model trained with normal observation sequences, vGuard can effectively detect abnormal behaviors online. vGuard is a general framework that can train different types of VNF behavior models. We implement the vGuard prototype in the OpenStack platform. Two types of VNF models, virtual router and virtual firewall, are trained using real normal network traffics in our experiment evaluation. A collection of abnormal attack cases are tested on the VNFs that showed the effectiveness of vGuard in detecting VNF behavior anomalies.

[1]  Sung-Bae Cho,et al.  Efficient anomaly detection by modeling privilege flows using hidden Markov model , 2003, Comput. Secur..

[2]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[3]  Leyla Bilge,et al.  Exposure: A Passive DNS Analysis Service to Detect and Report Malicious Domains , 2014, TSEC.

[4]  Abhinav Srivastava,et al.  Credit Card Fraud Detection Using Hidden Markov Model , 2008, IEEE Transactions on Dependable and Secure Computing.

[5]  Salem Benferhat,et al.  A Naive Bayes Approach for Detecting Coordinated Attacks , 2008, 2008 32nd Annual IEEE International Computer Software and Applications Conference.

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  Mohammed J. Zaki,et al.  ADMIT: anomaly-based data mining for intrusions , 2002, KDD.

[8]  Xinghuo Yu,et al.  A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection , 2009, IEEE Network.

[9]  L. Baum,et al.  Growth transformations for functions on manifolds. , 1968 .

[10]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[11]  James Cannady,et al.  Artificial Neural Networks for Misuse Detection , 1998 .

[12]  Shu-Chin Wang,et al.  An Integrated Intrusion Detection System for Cluster-based Wireless Sensor Networks , 2011, Expert Syst. Appl..

[13]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[14]  Roberto Tronci,et al.  HMMPayl: An intrusion detection system based on Hidden Markov Models , 2011, Comput. Secur..

[15]  Zied Elouedi,et al.  Naive Bayes vs decision trees in intrusion detection systems , 2004, SAC '04.

[16]  Filip De Turck,et al.  Network Function Virtualization: State-of-the-Art and Research Challenges , 2015, IEEE Communications Surveys & Tutorials.

[17]  Christopher Krügel,et al.  Bayesian event classification for intrusion detection , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[18]  M. Sadiq Ali Khan,et al.  Rule based Network Intrusion Detection using Genetic Algorithm , 2011 .

[19]  Andrew H. Sung,et al.  Intrusion detection using an ensemble of intelligent paradigms , 2005, J. Netw. Comput. Appl..

[20]  Vipin Kumar,et al.  Anomaly Detection for Discrete Sequences: A Survey , 2012, IEEE Transactions on Knowledge and Data Engineering.

[21]  Jiankun Hu,et al.  Modeling Oscillation Behavior of Network Traffic by Nested Hidden Markov Model with Variable State-Duration , 2013, IEEE Transactions on Parallel and Distributed Systems.

[22]  Yinhui Li,et al.  An efficient intrusion detection system based on support vector machines and gradually feature removal method , 2012, Expert Syst. Appl..

[23]  Nasser Yazdani,et al.  Mutual information-based feature selection for intrusion detection systems , 2011, J. Netw. Comput. Appl..

[24]  L. R. Rabiner,et al.  Some properties of continuous hidden Markov model representations , 1985, AT&T Technical Journal.

[25]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[26]  Taeshik Shon,et al.  A hybrid machine learning approach to network anomaly detection , 2007, Inf. Sci..

[27]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[28]  Carlos Martín-Vide,et al.  Evolutionary Design of Intrusion Detection Programs , 2007, Int. J. Netw. Secur..

[29]  Radu State,et al.  Machine Learning Approach for IP-Flow Record Anomaly Detection , 2011, Networking.

[30]  Vir V. Phoha,et al.  Investigating hidden Markov models capabilities in anomaly detection , 2005, ACM-SE 43.

[31]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[32]  Muttukrishnan Rajarajan,et al.  A survey of intrusion detection techniques in Cloud , 2013, J. Netw. Comput. Appl..

[33]  Sadok Ben Yahia,et al.  OMC-IDS: At the Cross-Roads of OLAP Mining and Intrusion Detection , 2012, PAKDD.

[34]  Robert Sabourin,et al.  On the memory complexity of the forward-backward algorithm , 2010, Pattern Recognit. Lett..

[35]  M. Rief,et al.  The Complex Folding Network of Single Calmodulin Molecules , 2011, Science.

[36]  Robert Sabourin,et al.  A survey of techniques for incremental learning of HMM parameters , 2012, Inf. Sci..

[37]  Chun-Hung Richard Lin,et al.  Intrusion detection system: A comprehensive review , 2013, J. Netw. Comput. Appl..

[38]  Jay L. Devore,et al.  Probability and statistics for engineering and the sciences , 1982 .

[39]  Robert K. Cunningham,et al.  Improving Intrusion Detection Performance using Keyword Selection and Neural Networks , 2000, Recent Advances in Intrusion Detection.

[40]  Misty K. Blowers,et al.  Machine Learning Applied to Cyber Operations , 2014, Network Science and Cybersecurity.

[41]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[42]  Boleslaw K. Szymanski,et al.  NETWORK-BASED INTRUSION DETECTION USING NEURAL NETWORKS , 2002 .

[43]  Shunzheng Yu,et al.  A General Collaborative Framework for Modeling and Perceiving Distributed Network Behavior , 2016, IEEE/ACM Transactions on Networking.

[44]  V. Rao Vemuri,et al.  Robust Support Vector Machines for Anomaly Detection in Computer Security , 2003, ICMLA.

[45]  Wei Lu,et al.  Detecting New Forms of Network Intrusion Using Genetic Programming , 2004, Comput. Intell..

[46]  Noorhaniza Wahid,et al.  A hybrid network intrusion detection system using simplified swarm optimization (SSO) , 2012, Appl. Soft Comput..