Cyber intrusion detection through association rule mining on multi-source logs

Security logs in cloud environment like intrusion detection system (IDS) logs, firewall logs, and system logs provide historical information describing potential security risks. However, the use of logs for cyber intrusion detection relies heavily on expert knowledge. It is very difficult for the non-expert to identify these intrusion behaviors. This paper proposes a new method for mining association rules from multi-source logs to detect various intrusion behaviors in the cloud computing platform. In this method, a rule base is constructed to detect cyber intrusion. An adaptive approach is used to speed up the calculation of the association rule mining, in which the decision depends on the time complexity of the algorithm. Various cyber-attacks are simulated in the verification experiments which show the calculation speed of the proposed method is faster than other algorithms. Furthermore, compared with other methods, the performance of the proposed intrusion detection method is better than others in term of precision, recall, and f-measure.

[1]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[2]  Won Hyung Park,et al.  A study on cyber threat prediction based on intrusion detection event for APT attack detection , 2012, Multimedia Tools and Applications.

[3]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.

[4]  Qiang Fu,et al.  Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[5]  Manuel López Martín,et al.  Adversarial environment reinforcement learning algorithm for intrusion detection , 2019, Comput. Networks.

[6]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[7]  Ke Zhang,et al.  Execution anomaly detection in large-scale systems through console log analysis , 2018, J. Syst. Softw..

[8]  Abderrahim Beni Hssane,et al.  Big healthcare data: preserving security and privacy , 2018, Journal of Big Data.

[9]  Kangfeng Zheng,et al.  An Explainable Machine Learning Framework for Intrusion Detection Systems , 2020, IEEE Access.

[10]  Jung-Shian Li,et al.  Novel intrusion prediction mechanism based on honeypot log similarity , 2016, Int. J. Netw. Manag..

[11]  Gulshan Kumar.  An improved ensemble approach for effective intrusion detection , 2019, The Journal of Supercomputing.

[12]  Zili Zhang,et al.  A MapReduce-Based Parallel Frequent Pattern Growth Algorithm for Spatiotemporal Association Analysis of Mobile Trajectory Big Data , 2018, Complex..

[13]  Naruemon Wattanapongsakorn,et al.  Web-based monitoring approach for network-based intrusion detection and prevention , 2014, Multimedia Tools and Applications.

[14]  Sanjay Ghemawat,et al.  MapReduce: simplified data processing on large clusters , 2008, CACM.

[15]  Yu-Lin He,et al.  Fuzziness based semi-supervised learning approach for intrusion detection system , 2017, Inf. Sci..

[16]  K. Muneeswaran,et al.  Firefly algorithm based feature selection for network intrusion detection , 2019, Comput. Secur..

[17]  Lin Chen,et al.  A Game Theoretical Framework on Intrusion Detection in Heterogeneous Networks , 2009, IEEE Transactions on Information Forensics and Security.

[18]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Network Intrusion Detection , 2004, RAID.

[19]  Sadok Ben Yahia,et al.  OMC-IDS: At the Cross-Roads of OLAP Mining and Intrusion Detection , 2012, PAKDD.

[20]  Francisco Herrera,et al.  Mining association rules on Big Data through MapReduce genetic programming , 2017, Integr. Comput. Aided Eng..

[21]  Ünal Çavusoglu,et al.  A new hybrid approach for intrusion detection using machine learning methods , 2019, Applied Intelligence.

[22]  Jonghyun Kim,et al.  Design of network threat detection and classification based on machine learning on cloud computing , 2018, Cluster Computing.

[23]  Simon Parkinson,et al.  Eliciting and utilising knowledge for security event log analysis: An association rule mining and automated planning approach , 2018, Expert Syst. Appl..

[24]  A. Piskozub,et al.  Real-Time Intrusion Prevention and Anomaly Analyze System for Corporate Networks , 2007, 2007 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications.

[25]  Elias Bou-Harb,et al.  Survey of Attack Projection, Prediction, and Forecasting in Cyber Security , 2019, IEEE Communications Surveys & Tutorials.

[26]  Salvatore J. Stolfo,et al.  Data Mining Approaches for Intrusion Detection , 1998, USENIX Security Symposium.

[27]  Yasir Saleem,et al.  Enhanced Network Intrusion Detection using Deep Convolutional Neural Networks , 2018, KSII Trans. Internet Inf. Syst..

[28]  Ali A. Ghorbani,et al.  Application of deep learning to cybersecurity: A survey , 2019, Neurocomputing.

[29]  Long Wang,et al.  LADRA: Log-based abnormal task detection and root-cause analysis in big data processing with Spark , 2019, Future Gener. Comput. Syst..

[30]  Aiko Pras,et al.  An Overview of IP Flow-Based Intrusion Detection , 2010, IEEE Communications Surveys & Tutorials.

[31]  Jing Ou,et al.  Network threat detection based on correlation analysis of multi-platform multi-source alert data , 2018, Multimedia Tools and Applications.

[32]  Sanjay Rathee,et al.  Adaptive-Miner: an efficient distributed association rule mining algorithm on Spark , 2018, Journal of Big Data.

[33]  Andrew H. Sung,et al.  Intrusion detection using an ensemble of intelligent paradigms , 2005, J. Netw. Comput. Appl..

[34]  Fadi Al-Turjman,et al.  An intrusion detection scheme based on the ensemble of discriminant classifiers , 2020, Comput. Electr. Eng..

[35]  Jun Yang,et al.  Improved traffic detection with support vector machine based on restricted Boltzmann machine , 2017, Soft Comput..

[36]  Wei Li,et al.  Using Genetic Algorithm for Network Intrusion Detection , 2004 .