Evasion-resistant network scan detection

Popular network scan detection algorithms operate through evaluating external sources for unusual connection patterns and traffic rates. Research has revealed evasive tactics that enable full circumvention of existing approaches (specifically the widely cited Threshold Random Walk algorithm). To prevent use of these circumvention techniques, we propose a novel approach to network scan detection that evaluates the behavior of internal network nodes, and combine it with other established techniques of scan detection. By itself, our algorithm is an efficient, protocol-agnostic, completely unsupervised method that requires no a priori knowledge of the network being defended beyond which hosts are internal and which hosts are external to the network, and is capable of detecting network scanning attempts regardless of the rate of the scan (working even with connectionless protocols). We demonstrate the effectiveness of our method on both live data from an enterprise-scale network and on simulated scan data, finding a false positive rate of just 0.000034% with respect to the number of inbound flows. When combined with both Threshold Random Walk and simple rate-limiting detection, we achieve an overall detection rate of 94.44%.

[1]  Hui Xiong,et al.  Scan Detection: A Data Mining Approach , 2006, SDM.

[2]  Sanjeev Khanna,et al.  Space-efficient online computation of quantile summaries , 2001, SIGMOD '01.

[3]  Kotagiri Ramamohanarao,et al.  A probabilistic approach to detecting network scans , 2002, NOMS 2002. IEEE/IFIP Network Operations and Management Symposium. ' Management Solutions for the New Communications World'(Cat. No.02CH37327).

[4]  Hari Balakrishnan,et al.  Fast portscan detection using sequential hypothesis testing , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[5]  Paul C. van Oorschot,et al.  Network scan detection with LQS: a lightweight, quick and stateful algorithm , 2011, ASIACCS '11.

[6]  Dawn Xiaodong Song,et al.  Distributed Evasive Scan Techniques and Countermeasures , 2007, DIMVA.

[7]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[8]  Jugal K. Kalita,et al.  Surveying Port Scans and Their Detection Methodologies , 2011, Comput. J..

[9]  Biswanath Mukherjee,et al.  A network security monitor , 1990, Proceedings. 1990 IEEE Computer Society Symposium on Research in Security and Privacy.

[10]  Roberto Baldoni,et al.  Inter-domain stealthy port scan detection through complex event processing , 2011, EWDC '11.

[11]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[12]  Stuart Staniford-Chen,et al.  Practical Automated Detection of Stealthy Portscans , 2002, J. Comput. Secur..

[13]  Felicita Di Giandomenico.  Proceedings of the 13th European Workshop on Dependable Computing, EWDC '11, Pisa, Italy, May 11-12, 2011 , 2011, EWDC.

[14]  Carrie Gates,et al.  Coordinated Scan Detection , 2009, NDSS.

[15]  Paul Barford,et al.  Intrusion as (anti)social communication: characterization and detection , 2012, KDD.

[16]  Yu Zhang,et al.  Allocation Schemes, Architectures, and Policies for Collaborative Port Scanning Attacks , 2011 .

[17]  Fabio Ricciato,et al.  Detecting Scanners: Empirical Assessment on a 3G Network , 2009, Int. J. Netw. Secur..

[18]  Richard E. Harang,et al.  Limitations to threshold random walk scan detection and mitigating enhancements , 2013, 2013 IEEE Conference on Communications and Network Security (CNS).

[19]  Mehiar Dabbagh,et al.  Slow port scanning detection , 2011, 2011 7th International Conference on Information Assurance and Security (IAS).

[20]  Tao Ye,et al.  Connectionless port scan detection on the backbone , 2006, 2006 IEEE International Performance Computing and Communications Conference.

[21]  Xinjia Chen.  New Sequential Methods for Detecting Portscanners , 2012, ArXiv.

[22]  Giuseppe Antonio Di Luna,et al.  Collaborative Detection of Coordinated Port Scans , 2013, ICDCN.