Distributed Processing of Snort Alert Log using Hadoop

Snort is a famous tool for Intrusion Detection System (IDS), which is used to gather and analyse network packet in order to decide attacks through network. Until now, although processing a number of warning messages in real time, Snort is executed mainly in single computer systems. Unfortunately, current amount of network messages exceeds processing capacity of single computer systems. In order to embrace the huge amount of network messages, we have constructed a distributed IDS using Hadoop, HDFS, and 8 working nodes. Experimental results show that our distributed IDS has 426% performance compared to a single computer system. Keyword-Intrusion Detection System, Snort, Distributed Framework, Hadoop, HDFS

[1]  Andy Konwinski,et al.  Chukwa: A large-scale monitoring system , 2008 .

[2]  Supra-linear Packet Processing Performance with Intel ® Multi-core Processors , 2022 .

[3]  Jianhua Yang,et al.  Exploring and Enhancing the Performance of Parallel IDS on Multi-core Processors , 2011, 2011IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications.

[4]  George Varghese,et al.  Applying Fast String Matching to Intrusion Detection , 2001 .

[5]  Alfred V. Aho,et al.  Efficient string matching , 1975, Commun. ACM.

[6]  Yung Ryn Choe,et al.  Conservative vs. Optimistic Parallelization of Stateful Network Intrusion Detection , 2007, ISPASS 2008 - IEEE International Symposium on Performance Analysis of Systems and software.

[7]  Wei-Yu Chen,et al.  ICAS: An inter-VM IDS Log Cloud Analysis System , 2011, 2011 IEEE International Conference on Cloud Computing and Intelligence Systems.

[8]  Nen-Fu Huang,et al.  A fast pattern matching algorithm for network processor-based intrusion detection system , 2004, IEEE International Conference on Performance, Computing, and Communications, 2004.

[9]  Youngseok Lee,et al.  A Hadoop-Based Packet Trace Processing Tool , 2011, TMA.

[10]  C.J. Coit,et al.  Towards faster string matching for intrusion detection or exceeding the speed of Snort , 2001, Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01.

[11]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).