Contextual Anomaly Detection Methods for Addressing Intrusion Detection

One promising method to detect cyber-crime is anomaly detection, which enables one to detect new, unseen attacks. Despite this ability, anomaly detection methods only have limited utilization in practice, due to the high number of false alarms generated. Recent research has shown that the number of false alarms can be reduced drastically by considering the context in which these alarms occur. However, important questions include, What does context mean in the realm of anomaly detection? and How can it be incorporated to identify potential cyber-crime? To address these questions, this chapter provides novel definitions of context and contextual anomaly detection methods. Based on these, a new taxonomy is proposed for contextual anomaly detection methods, which organizes the methods by the specific problems they address. Further, the chapter highlights the potential of contextual anomaly detection for the reduction of false alarms, particularly for network anomaly detection and provides an introduction and holistic overview of the field for professionals and researchers. Contextual Anomaly Detection Methods for Addressing Intrusion Detection

[1]  Jill Slay,et al.  Novel Geometric Area Analysis Technique for Anomaly Detection Using Trapezoidal Area Estimation on Large-Scale Networks , 2019, IEEE Transactions on Big Data.

[2]  Jugal K. Kalita,et al.  An effective unsupervised network anomaly detection method , 2012, ICACCI '12.

[3]  Xizhao Wang,et al.  Covariance-Matrix Modeling and Detecting Various Flooding Attacks , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[4]  Vyas Sekar,et al.  An empirical evaluation of entropy-based traffic anomaly detection , 2008, IMC '08.

[5]  Patrick Brézillon,et al.  Context in problem solving: a survey , 1999, The Knowledge Engineering Review.

[6]  Carlos García Garino,et al.  Automatic network intrusion detection: Current techniques and open issues , 2012, Comput. Electr. Eng..

[7]  Sushma Jain,et al.  Hybrid Genetic Fuzzy Rule Based Inference Engine to Detect Intrusion in Networks , 2014, ISI.

[8]  Sanjay Ranka,et al.  Conditional Anomaly Detection , 2007, IEEE Transactions on Knowledge and Data Engineering.

[9]  Jill Slay,et al.  Anomaly Detection System Using Beta Mixture Models and Outlier Detection , 2018 .

[10]  Xiangjian He,et al.  Intrusion detection method based on nonlinear correlation measure , 2014, Int. J. Internet Protoc. Technol..

[11]  Jugal K. Kalita,et al.  Network Anomaly Detection: Methods, Systems and Tools , 2014, IEEE Communications Surveys & Tutorials.

[12]  Kensuke Fukuda,et al.  Non-linear regression for bivariate self-similarity identification — application to anomaly detection in Internet traffic based on a joint scaling analysis of packet and byte counts , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Weiru Liu,et al.  Natural Laws as a Baseline for Network Anomaly Detection , 2016, 2016 IEEE Trustcom/BigDataSE/ISPA.

[14]  Taghi M. Khoshgoftaar,et al.  Intrusion detection and Big Heterogeneous Data: a Survey , 2015, Journal of Big Data.

[15]  Yizhou Sun,et al.  On community outliers and their efficient detection in information networks , 2010, KDD.

[16]  Qiang Chen,et al.  Multivariate Statistical Analysis of Audit Trails for Host-Based Intrusion Detection , 2002, IEEE Trans. Computers.

[17]  Tai-hoon Kim,et al.  Linear Correlation-Based Feature Selection for Network Intrusion Detection Model , 2013, SecNet.

[18]  Biming Tian,et al.  Anomaly detection in wireless sensor networks: A survey , 2011, J. Netw. Comput. Appl..

[19]  George Karabatis,et al.  Beyond data: contextual information fusion for cyber security analytics , 2016, SAC.

[20]  Xiao Qin,et al.  A relevant subspace based contextual outlier mining algorithm , 2016, Knowl. Based Syst..

[21]  S. T. Sarasamma,et al.  Hierarchical Kohonenen net for anomaly detection in network security , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Milos Hauskrecht,et al.  Conditional Anomaly Detection with Soft Harmonic Functions , 2011, 2011 IEEE 11th International Conference on Data Mining.

[23]  Jill Slay,et al.  Big Data Analytics for Intrusion Detection System: Statistical Decision-Making Using Finite Dirichlet Mixture Models , 2017 .

[24]  Dong Xiang,et al.  Information-theoretic measures for anomaly detection , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[25]  Mohiuddin Ahmed,et al.  Network traffic analysis based on collective anomaly detection , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[26]  Jonathon A. Chambers,et al.  Adding contextual information to Intrusion Detection Systems using Fuzzy Cognitive Maps , 2016, 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA).

[27]  George Karabatis,et al.  Contextual information fusion for intrusion detection: a survey and taxonomy , 2017, Knowledge and Information Systems.

[28]  Roberto Battiti,et al.  Identifying intrusions in computer networks with principal component analysis , 2006, First International Conference on Availability, Reliability and Security (ARES'06).

[29]  José M. Fernandez,et al.  Semantic-based context-aware alert fusion for distributed Intrusion Detection Systems , 2013, 2013 International Conference on Risks and Security of Internet and Systems (CRiSIS).

[30]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[31]  Elizabeth Chang,et al.  Cyber Situational Awareness for CPS, 5G and IoT , 2017 .

[32]  Achim P. Karduck,et al.  SIM in light of big data , 2015, 2015 11th International Conference on Innovations in Information Technology (IIT).

[33]  M. Shyu,et al.  A Novel Anomaly Detection Scheme Based on Principal Component Classifier , 2003 .

[34]  Lauro Snidaro,et al.  Context-Enhanced Information Fusion , 2016, Advances in Computer Vision and Pattern Recognition.

[35]  Srinivasan Parthasarathy,et al.  Robust Contextual Outlier Detection: Where Context Meets Sparsity , 2016, CIKM.

[36]  Bill N. Schilit,et al.  Context-aware computing applications , 1994, Workshop on Mobile Computing Systems and Applications.

[37]  Jill Slay,et al.  The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set , 2016, Inf. Secur. J. A Glob. Perspect..

[38]  Jian Ma,et al.  A new approach to intrusion detection using Artificial Neural Networks and fuzzy clustering , 2010, Expert Syst. Appl..

[39]  Ian Davidson,et al.  Discovering Contexts and Contextual Outliers Using Random Walks in Graphs , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[40]  Samuel Kounev,et al.  Evaluating Computer Intrusion Detection Systems , 2015, ACM Comput. Surv..

[41]  Nour Moustafa,et al.  UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set) , 2015, 2015 Military Communications and Information Systems Conference (MilCIS).

[42]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[43]  Tharam S. Dillon,et al.  CorrCorr: A feature selection method for multivariate correlation network anomaly detection techniques , 2019, Comput. Secur..

[44]  Jennifer E. Rowley,et al.  The wisdom hierarchy: representations of the DIKW hierarchy , 2007, J. Inf. Sci..

[45]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[46]  Shouhuai Xu,et al.  Spatiotemporal Patterns and Predictability of Cyberattacks , 2015, PloS one.

[47]  Karl N. Levitt,et al.  GrIDS A Graph-Based Intrusion Detection System for Large Networks , 1996 .

[48]  Stefan Axelsson,et al.  Intrusion Detection Systems: A Survey and Taxonomy , 2002 .

[49]  Xiangjian He,et al.  Building an Intrusion Detection System Using a Filter-Based Feature Selection Algorithm , 2016, IEEE Transactions on Computers.

[50]  Philip K. Chan,et al.  PHAD: packet header anomaly detection for identifying hostile network traffic , 2001 .

[51]  Aiko Pras,et al.  An Overview of IP Flow-Based Intrusion Detection , 2010, IEEE Communications Surveys & Tutorials.

[52]  Jaideep Srivastava,et al.  Contextual Anomaly Detection in Text Data , 2012, Algorithms.

[53]  Bernhard Pfahringer,et al.  Winning the KDD99 classification cup: bagged boosting , 2000, SKDD.

[54]  Charu C. Aggarwal,et al.  Outlier Detection for Temporal Data: A Survey , 2014, IEEE Transactions on Knowledge and Data Engineering.

[55]  Wenlong Fu,et al.  A Neural Network Based Intrusion Detection Data Fusion Model , 2010, 2010 Third International Joint Conference on Computational Science and Optimization.

[56]  James Bailey,et al.  Mining multidimensional contextual outliers from categorical relational data , 2013, SSDBM.

[57]  Charu C. Aggarwal Spatial Outlier Detection , 2013 .

[58]  Fabio Roli,et al.  Information fusion for computer security: State of the art and open issues , 2009, Inf. Fusion.

[59]  Marcin Szpyrka,et al.  An Entropy-Based Network Anomaly Detection Method , 2015, Entropy.

[60]  Hans-Peter Kriegel,et al.  Outlier Detection in Arbitrarily Oriented Subspaces , 2012, 2012 IEEE 12th International Conference on Data Mining.

[61]  Li Guo,et al.  Survey and Taxonomy of Feature Selection Algorithms in Intrusion Detection System , 2006, Inscrypt.

[62]  Hsin-Hui Chiu,et al.  CEO Bonus Pay and Firm Credit Risk , 2020, International Journal of Risk and Contingency Management.

[63]  Simone A. Ludwig Intrusion detection of multiple attack classes using a deep neural net ensemble , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[64]  Heiko Paulheim,et al.  A decomposition of the outlier detection problem into a set of supervised learning problems , 2015, Machine Learning.

[65]  Julia Kiseleva,et al.  Context mining and integration into predictive web analytics , 2013, WWW.

[66]  Xiangjian He,et al.  A System for Denial-of-Service Attack Detection Based on Multivariate Correlation Analysis , 2011, IEEE Transactions on Parallel and Distributed Systems.

[67]  James Llinas,et al.  An introduction to multisensor data fusion , 1997, Proc. IEEE.

[68]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[69]  M. Otto,et al.  Outliers in Time Series , 1972 .

[70]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[71]  Miriam A. M. Capretz,et al.  Contextual anomaly detection framework for big sensor data , 2015, Journal of Big Data.

[72]  D. S. Yeung,et al.  Network intrusion detection in covariance feature space , 2007, Pattern Recognit..

[73]  Xiangjian He,et al.  A System for Denial-of-Service Attack Detection Based on Multivariate Correlation Analysis , 2014, IEEE Transactions on Parallel and Distributed Systems.

[74]  Mohiuddin Ahmed,et al.  A survey of network anomaly detection techniques , 2016, J. Netw. Comput. Appl..

[75]  Sui Song,et al.  Flow-based Statistical Aggregation Schemes for Network Anomaly Detection , 2006, 2006 IEEE International Conference on Networking, Sensing and Control.