Behavioral Analysis of System Call Sequences Using LSTM Seq-Seq, Cosine Similarity and Jaccard Similarity for Real-Time Anomaly Detection

With the advent of technology, sophisticated malware presents a significant threat to computer security. In this work, we propose anomaly detection techniques that learn three different behaviors of windows system-call sequences. We apply Long-Short-Term-Memory (LSTM) for temporal behavior, Cosine Similarity for frequency distribution behavior, and Jaccard Similarity for commonality behavior. The proposed framework monitors the processes in a hypervisor-based environment to detect compromised virtual machines. System call sequences of normal processes and malware-infected processes were extracted with memory forensic techniques. Our proposed anomaly detection techniques were able to learn the above three behavior of the system call sequences with 99% accuracy.

[1]  Jennia Hizver,et al.  Real-time deep virtual machine introspection and its applications , 2014, VEE '14.

[2]  Hervé Debar,et al.  A neural network component for an intrusion detection system , 1992, Proceedings 1992 IEEE Computer Society Symposium on Research in Security and Privacy.

[3]  Ralf C. Staudemeyer,et al.  Applying long short-term memory recurrent neural networks to intrusion detection , 2015 .

[4]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[5]  D. B. Choksi,et al.  IMPLEMENTATION OF PROCESS FORENSIC FOR SYSTEM CALLS , 2014 .

[6]  Ralf C. Staudemeyer,et al.  Evaluating performance of long short-term memory recurrent neural networks on intrusion detection data , 2013, SAICSIT '13.

[7]  S.S. Iyengar,et al.  Learning-Based Model to Fight against Fake Like Clicks on Instagram Posts , 2019, 2019 SoutheastCon.

[8]  Jian Ma,et al.  A new approach to intrusion detection using Artificial Neural Networks and fuzzy clustering , 2010, Expert Syst. Appl..

[9]  Christopher Krügel,et al.  A survey on automated dynamic malware-analysis techniques and tools , 2012, CSUR.

[10]  Sung-Bae Cho,et al.  Efficient anomaly detection by modeling privilege flows using hidden Markov model , 2003, Comput. Secur..

[11]  Étienne Payet,et al.  Static Analysis of Android Programs , 2011, CADE.

[12]  Stephen D. Wolthusen,et al.  Anomaly Detection for Ephemeral Cloud IaaS Virtual Machines , 2013, NSS.

[13]  Jayesh Soni.  Effective Machine Learning Approach to Detect Groups of Fake Reviewers , 2018 .

[14]  Andrew H. Sung,et al.  Intrusion detection using neural networks and support vector machines , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[15]  Yanfang Ye,et al.  IMDS: intelligent malware detection system , 2007, KDD '07.

[16]  Wei Wang,et al.  Modeling program behaviors by hidden Markov models for intrusion detection , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[18]  Risto Miikkulainen,et al.  Intrusion Detection with Neural Networks , 1997, NIPS.

[19]  Jiankun Hu,et al.  A Semantic Approach to Host-Based Intrusion Detection Systems Using Contiguousand Discontiguous System Call Patterns , 2014, IEEE Transactions on Computers.

[20]  Bhavani M. Thuraisingham,et al.  A scalable multi-level feature extraction technique to detect malicious executables , 2007, Inf. Syst. Frontiers.