A lightweight web server anomaly detection method based on transductive scheme and genetic algorithms

World Wide Web (WWW) is one of the most popular applications currently running on the Internet and web server is a crucial component for this application. However, network anomalies especially Distributed Denial-of-Service (DDoS) attacks bombard web server, degrade its Quality of Service (QoS) and even deny the legitimate users' requests. Traditional network anomaly detection methods often lead to high false positives and expensive computational cost, thus unqualified for real-time web server anomaly detection. To solve these problems, in this paper we first propose an efficient network anomaly detection method based on Transductive Confidence Machines for K-Nearest Neighbors (TCM-KNN) algorithm. Secondly, we integrate a lot of objective and efficient anomalies impact metrics from the perceptions of the end users into TCM-KNN algorithm to build a robust web sever anomaly detection mechanism. Finally, Genetic Algorithm (GA) based instance selection method is introduced to boost the real-time detection performance of our method. We evaluate our method on a series of experiments both on well-known KDD Cup 1999 dataset and concrete dataset collected from real network traffic. The results demonstrate our methods are actually effective and lightweight for real-time web server anomaly detection.

[1]  Sushil Jajodia,et al.  Applications of Data Mining in Computer Security , 2002, Advances in Information Security.

[2]  Daniel Barbará,et al.  Detecting outliers using transduction and statistical testing , 2006, KDD '06.

[3]  Alexander Gammerman,et al.  Prediction algorithms and confidence measures based on algorithmic randomness theory , 2002, Theor. Comput. Sci..

[4]  Salvatore J. Stolfo,et al.  Data Mining Approaches for Intrusion Detection , 1998, USENIX Security Symposium.

[5]  Sonia Fahmy,et al.  Measuring denial Of service , 2006, QoP '06.

[6]  Li Guo,et al.  Network anomaly detection based on TCM-KNN algorithm , 2007, ASIACCS '07.

[7]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[8]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[9]  Eleazar Eskin,et al.  A GEOMETRIC FRAMEWORK FOR UNSUPERVISED ANOMALY DETECTION: DETECTING INTRUSIONS IN UNLABELED DATA , 2002 .

[10]  Kotagiri Ramamohanarao,et al.  Protection from distributed denial of service attacks using history-based IP filtering , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[11]  Philip K. Chan,et al.  Learning nonstationary models of normal network traffic for detecting novel attacks , 2002, KDD.

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  Dorothy E. Denning,et al.  An Intrusion-Detection Model , 1986, 1986 IEEE Symposium on Security and Privacy.

[14]  Mark Claypool,et al.  The effects of loss and latency on user performance in unreal tournament 2003® , 2004, NetGames '04.

[15]  Francisco Herrera,et al.  Using evolutionary algorithms as instance selection for data reduction in KDD: an experimental study , 2003, IEEE Trans. Evol. Comput..

[16]  Wei Li,et al.  Using Genetic Algorithm for Network Intrusion Detection , 2004 .

[17]  David K. Y. Yau,et al.  Defending against distributed denial-of-service attacks with max-min fair server-centric router throttles , 2005, IEEE/ACM Transactions on Networking.

[18]  Kang G. Shin,et al.  Detecting SYN flooding attacks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[19]  Li Guo,et al.  An active learning based TCM-KNN algorithm for supervised network intrusion detection , 2007, Comput. Secur..

[20]  E. Talbi,et al.  A Genetic Algorithm for Feature Selection in Data-Mining for Genetics , 2001 .

[21]  Kang G. Shin,et al.  Defense Against Spoofed IP Traffic Using Hop-Count Filtering , 2007, IEEE/ACM Transactions on Networking.

[22]  Anup K. Ghosh,et al.  A Study in Using Neural Networks for Anomaly and Misuse Detection , 1999, USENIX Security Symposium.

[23]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 1997, Texts in Computer Science.