HMMPayl: An intrusion detection system based on Hidden Markov Models

Nowadays the security of Web applications is one of the key topics in Computer Security. Among all the solutions that have been proposed so far, the analysis of the HTTP payload at the byte level has proven to be effective as it does not require the detailed knowledge of the applications running on the Web server. The solutions proposed in the literature actually achieved good results for the detection rate, while there is still room for reducing the false positive rate. To this end, in this paper we propose HMMPayl, an IDS where the payload is represented as a sequence of bytes, and the analysis is performed using Hidden Markov Models (HMM). The algorithm we propose for feature extraction and the joint use of HMM guarantee the same expressive power of n -gram analysis, while allowing to overcome its computational complexity. In addition, we designed HMMPayl following the Multiple Classifiers System paradigm to provide for a better classification accuracy, to increase the difficulty of evading the IDS, and to mitigate the weaknesses due to a non optimal choice of HMM parameters. Experimental results, obtained both on public and private datasets, show that the analysis performed by HMMPayl is particularly effective against the most frequent attacks toward Web applications (such as XSS and SQL-Injection). In particular, for a fixed false positive rate, HMMPayl achieves a higher detection rate respect to previously proposed approaches it has been compared with.

[1]  Robert P. W. Duin,et al.  The combining classifier: to train or not to train? , 2002, Object recognition supported by user interaction for service robots.

[2]  Barak A. Pearlmutter,et al.  Detecting intrusions using system calls: alternative data models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[3]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[4]  Wenke Lee,et al.  McPAD: A multiple classifier system for accurate payload-based anomaly detection , 2009, Comput. Networks.

[5]  Ching Y. Suen,et al.  n-Gram Statistics for Natural Language Understanding and Text Processing , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Giorgio Giacinto,et al.  HMM-Web: A Framework for the Detection of Attacks Against Web Applications , 2009, 2009 IEEE International Conference on Communications.

[7]  Fabio Roli,et al.  Dynamic Score Selection for Fusion of Multiple Biometric Matchers , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[8]  Salvatore J. Stolfo,et al.  Using artificial anomalies to detect unknown and known network intrusions , 2003, Knowledge and Information Systems.

[9]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[10]  Christopher Krügel,et al.  Service specific anomaly detection for network intrusion detection , 2002, SAC '02.

[11]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Network Intrusion Detection , 2004, RAID.

[12]  M Damashek,et al.  Gauging Similarity with n-Grams: Language-Independent Categorization of Text , 1995, Science.

[13]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[14]  Aleksander Kolcz,et al.  Feature Weighting for Improved Classifier Robustness , 2009, CEAS 2009.

[15]  Christopher Krügel,et al.  Protecting a Moving Target: Addressing Web Application Concept Drift , 2009, RAID.

[16]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Worm Detection and Signature Generation , 2005, RAID.

[17]  Debin Gao,et al.  Beyond Output Voting: Detecting Compromised Replicas Using HMM-Based Behavioral Distance , 2009, IEEE Transactions on Dependable and Secure Computing.

[18]  Gregory J. Conti,et al.  Toward Instrumenting Network Warfare Competitions to Generate Labeled Datasets , 2009, CSET.

[19]  Joshua Mason,et al.  English shellcode , 2009, CCS.

[20]  Hajime Inoue,et al.  Comparing Anomaly Detection Techniques for HTTP , 2007, RAID.

[21]  Christopher Krügel,et al.  Anomaly detection of web-based attacks , 2003, CCS '03.

[22]  Taku Komura,et al.  NiceMeetVR: facing professional baseball pitchers in the virtual batting cage , 2002, SAC '02.

[23]  Fabio Roli,et al.  Information fusion for computer security: State of the art and open issues , 2009, Inf. Fusion.

[24]  Salvatore J. Stolfo,et al.  Anagram: A Content Anomaly Detector Resistant to Mimicry Attack , 2006, RAID.

[25]  L. Baum,et al.  Growth transformations for functions on manifolds. , 1968 .

[26]  Fabio Roli,et al.  Adversarial Pattern Classification Using Multiple Classifiers and Randomisation , 2008, SSPR/SPR.

[27]  Matthew V. Mahoney,et al.  Network traffic anomaly detection based on packet bytes , 2003, SAC '03.

[28]  Venu Govindaraju,et al.  Robustness of multimodal biometric fusion methods against spoof attacks , 2009, J. Vis. Lang. Comput..

[29]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[30]  T. Speed,et al.  Biological Sequence Analysis , 1998 .

[31]  Sung-Bae Cho,et al.  Two Sophisticated Techniques to Improve HMM-Based Intrusion Detection Systems , 2003, RAID.

[32]  Salvatore J. Stolfo,et al.  Spectrogram: A Mixture-of-Markov-Chains Model for Anomaly Detection in Web Traffic , 2009, NDSS.

[33]  Richard Lippmann,et al.  The 1999 DARPA off-line intrusion detection evaluation , 2000, Comput. Networks.

[34]  Fabio Roli,et al.  Multiple Classifier Systems for Adversarial Classification Tasks , 2009, MCS.

[35]  Ludmila I. Kuncheva,et al.  Fuzzy Classifier Design , 2000, Studies in Fuzziness and Soft Computing.

[36]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[37]  Horst Bunke,et al.  Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[38]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[39]  F. Mora-Camino,et al.  Studies in Fuzziness and Soft Computing , 2011 .

[40]  Mehryar Mohri,et al.  Confidence Intervals for the Area Under the ROC Curve , 2004, NIPS.

[41]  Guofei Gu,et al.  Using an Ensemble of One-Class SVM Classifiers to Harden Payload-based Anomaly Detection Systems , 2006, Sixth International Conference on Data Mining (ICDM'06).

[42]  Wenke Lee,et al.  Evading network anomaly detection systems: formal reasoning and practical techniques , 2006, CCS '06.