Detection of slow port scans in flow-based network traffic

Frequently, port scans are early indicators of more serious attacks. Unfortunately, the detection of slow port scans in company networks is challenging due to the massive amount of network data. This paper proposes an innovative approach for preprocessing flow-based data which is specifically tailored to the detection of slow port scans. The preprocessing chain generates new objects based on flow-based data aggregated over time windows while taking domain knowledge as well as additional knowledge about the network structure into account. The computed objects are used as input for the further analysis. Based on these objects, we propose two different approaches for detection of slow port scans. One approach is unsupervised and uses sequential hypothesis testing whereas the other approach is supervised and uses classification algorithms. We compare both approaches with existing port scan detection algorithms on the flow-based CIDDS-001 data set. Experiments indicate that the proposed approaches achieve better detection rates and exhibit less false alarms than similar algorithms.

[1]  Jugal K. Kalita,et al.  Network Anomaly Detection: Methods, Systems and Tools , 2014, IEEE Communications Surveys & Tutorials.

[2]  Andreas Hotho,et al.  Flow-based benchmark data sets for intrusion detection , 2017 .

[3]  Andreas Hotho,et al.  A Toolset for Intrusion and Insider Threat Detection , 2017 .

[4]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[5]  Gordon Fyodor Lyon,et al.  Nmap Network Scanning: The Official Nmap Project Guide to Network Discovery and Security Scanning , 2009 .

[6]  Binxing Fang,et al.  A Novel Approach to Scan Detection on the Backbone , 2009, 2009 Sixth International Conference on Information Technology: New Generations.

[7]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[8]  Vyas Sekar,et al.  An empirical evaluation of entropy-based traffic anomaly detection , 2008, IMC '08.

[9]  Michel Cukier,et al.  An Improved Method for Anomaly-Based Network Scan Detection , 2015, SecureComm.

[10]  Maurizio Dusi,et al.  Estimating routing symmetry on single links by passive flow measurements , 2010, IWCMC.

[11]  Stuart Staniford-Chen,et al.  Practical Automated Detection of Stealthy Portscans , 2002, J. Comput. Secur..

[12]  Benoit Claise,et al.  Cisco Systems NetFlow Services Export Version 9 , 2004, RFC.

[13]  Lalu Banoth,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2017 .

[14]  Kensuke Fukuda,et al.  MAWILab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking , 2010, CoNEXT.

[15]  Tao Ye,et al.  Connectionless port scan detection on the backbone , 2006, 2006 IEEE International Performance Computing and Communications Conference.

[16]  Sumeet Dua,et al.  Data Mining and Machine Learning in Cybersecurity , 2011 .

[17]  George Karabatis,et al.  Queryable Semantics to Detect Cyber-Attacks: A Flow-Based Detection Approach , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[18]  Mourad Debbabi,et al.  Cyber Scanning: A Comprehensive Survey , 2014, IEEE Communications Surveys & Tutorials.

[19]  Vanessa Hertzog,et al.  Counter Hack Reloaded A Step By Step Guide To Computer Attacks And Effective Defenses , 2016 .

[20]  Alejandro Zunino,et al.  An empirical comparison of botnet detection methods , 2014, Comput. Secur..

[21]  Brett J. Borghetti,et al.  A Survey of Distance and Similarity Measures Used Within Network Intrusion Anomaly Detection , 2015, IEEE Communications Surveys & Tutorials.

[22]  Jian Pei,et al.  Data Mining: Concepts and Techniques, 3rd edition , 2006 .

[23]  Hari Balakrishnan,et al.  Fast portscan detection using sequential hypothesis testing , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[24]  Andreas Hotho,et al.  IP2Vec: Learning Similarities Between IP Addresses , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[25]  Benoit Claise,et al.  Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information , 2008, RFC.

[26]  Jugal K. Kalita,et al.  Surveying Port Scans and Their Detection Methodologies , 2011, Comput. J..

[27]  Joseph B. Kadane,et al.  Scan Detection on Very Large Networks Using Logistic Regression Modeling , 2006, 11th IEEE Symposium on Computers and Communications (ISCC'06).

[28]  Dhruba K. Bhattacharyya,et al.  Network Anomaly Detection: A Machine Learning Perspective , 2013 .

[29]  Gaoming Yang,et al.  An intrusion detection algorithm for sensor network based on normalized cut spectral clustering , 2019, PloS one.

[30]  Ali A. Ghorbani,et al.  Toward developing a systematic approach to generate benchmark datasets for intrusion detection , 2012, Comput. Secur..

[31]  Dieter Landes,et al.  Identifying Suspicious Activities in Company Networks Through Data Mining and Visualization , 2013 .