Enhancing collaborative intrusion detection via disagreement-based semi-supervised learning in IoT environments

Abstract Collaborative intrusion detection systems (CIDSs) are developing to improve the detection performance of a single detector in Internet of Things (IoT) networks, through exchanging and sharing data. For anomaly detection, machine learning is an important and essential tool to help identify the deviation between current events and pre-built profile. For a traditional supervised learning classifier, there is a need to provide training examples with ground-truth labels in advance. However, labeled instances are quite limited in real-world IoT scenarios, while unlabeled data/instances are widely available. This is because data labeling is a very expensive process that requires huge human efforts and knowledge inputs. To mitigate this issue, the use of semi-supervised learning algorithms is a promising solution, which can leverage unlabeled data to label data automatically without human intervention. In this work, we focus on semi-supervised learning and design DAS-CIDS, by applying disagreement-based semi-supervised learning algorithm for CIDSs. In the evaluation, we investigate the performance of DAS-CIDS using both datasets and in real IoT network environments, in the aspects of both detection performance and false alarm reduction. The experimental results show that as compared with traditional supervised classifiers, our approach is more effective in detecting intrusions and reducing false alarms by automatically leveraging unlabeled data.

[1]  Rayid Ghani,et al.  Analyzing the effectiveness and applicability of co-training , 2000, CIKM '00.

[2]  Horace Ho-Shing Ip,et al.  PMFA: Toward Passive Message Fingerprint Attacks on Challenge-Based Collaborative Intrusion Detection Networks , 2016, NSS.

[3]  Tao Xiang,et al.  A training-integrity privacy-preserving federated learning scheme with trusted execution environment , 2020, Inf. Sci..

[4]  Tsuhan Chen,et al.  Semi-supervised co-training and active learning based approach for multi-view intrusion detection , 2009, SAC '09.

[5]  Wenjuan Li,et al.  Improving the Performance of Neural Networks with Random Forest in Detecting Network Intrusions , 2013, ISNN.

[6]  Rich Caruana,et al.  Ensemble selection from libraries of models , 2004, ICML.

[7]  Yan Li,et al.  Design and Evaluation of Advanced Collusion Attacks on Collaborative Intrusion Detection Networks in Practice , 2016, 2016 IEEE Trustcom/BigDataSE/ISPA.

[8]  Nahid Shahmehri,et al.  A Trust-Aware, P2P-Based Overlay for Intrusion Detection , 2006, 17th International Workshop on Database and Expert Systems Applications (DEXA'06).

[9]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[10]  David J. Miller,et al.  A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data , 1996, NIPS.

[11]  Wenjuan Li,et al.  SOOA: Exploring Special On-Off Attacks on Challenge-Based Collaborative Intrusion Detection Networks , 2017, GPC.

[12]  Man Ho Au,et al.  Towards Statistical Trust Computation for Medical Smartphone Networks Based on Behavioral Profiling , 2017, IFIPTM.

[13]  David A. Landgrebe,et al.  The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon , 1994, IEEE Trans. Geosci. Remote. Sens..

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[16]  Stefan Axelsson,et al.  The base-rate fallacy and the difficulty of intrusion detection , 2000, TSEC.

[17]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[18]  Hervé Debar,et al.  A serial combination of anomaly and misuse IDSes applied to HTTP traffic , 2004, 20th Annual Computer Security Applications Conference.

[19]  Hakim Weatherspoon,et al.  Netbait: a Distributed Worm Detection Service , 2003 .

[20]  Raouf Boutaba,et al.  Trust Management for Host-Based Collaborative Intrusion Detection , 2008, DSOM.

[21]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.

[22]  Wenjuan Li,et al.  Design of intelligent KNN-based alarm filter using knowledge-based alert verification in intrusion detection , 2015, Secur. Commun. Networks.

[23]  Wenjuan Li,et al.  Design of Intrusion Sensitivity-Based Trust Management Model for Collaborative Intrusion Detection Networks , 2014, IFIPTM.

[24]  Zhi-Hua Zhou,et al.  Unlabeled Data and Multiple Views , 2011, PSL.

[25]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[26]  Marius Kloft,et al.  Active learning for network intrusion detection , 2009, AISec '09.

[27]  Lam-for Kwok,et al.  Enhancing False Alarm Reduction Using Voted Ensemble Selection in Intrusion Detection , 2013, Int. J. Comput. Intell. Syst..

[28]  Tadeusz Pietraszek,et al.  Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection , 2004, RAID.

[29]  Jun Zhang,et al.  JFCGuard: Detecting juice filming charging attack via processor usage analysis on smartphones , 2017, Comput. Secur..

[30]  Mahmoud Ammar,et al.  Journal of Information Security and Applications , 2022 .

[31]  Karen A. Scarfone,et al.  Guide to Intrusion Detection and Prevention Systems (IDPS) , 2007 .

[32]  Scott Shenker,et al.  The Architecture of PIER: an Internet-Scale Query Processor , 2005, CIDR.

[33]  Lam-For Kwok,et al.  Adaptive False Alarm Filter Using Machine Learning in Intrusion Detection , 2011 .

[34]  Giovanni Vigna,et al.  NetSTAT: a network-based intrusion detection approach , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[35]  Terran Lane,et al.  A Decision-Theoritic, Semi-Supervised Model for Intrusion Detection , 2006 .

[36]  Kim-Kwang Raymond Choo,et al.  A bayesian inference-based detection mechanism to defend medical smartphone networks against insider attacks , 2017, J. Netw. Comput. Appl..

[37]  Byung-Seo Kim,et al.  Internet of Things (IoT) Operating Systems Support, Networking Technologies, Applications, and Challenges: A Comparative Review , 2018, IEEE Communications Surveys & Tutorials.

[38]  Yingjie Tian,et al.  Semi-supervised learning methods for network intrusion detection , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[39]  Yuh-Jye Lee,et al.  Semi-supervised Learning for False Alarm Reduction , 2010, ICDM.

[40]  Heejo Lee,et al.  Group-Based Trust Management Scheme for Clustered Wireless Sensor Networks , 2009, IEEE Transactions on Parallel and Distributed Systems.

[41]  Jun Zhang,et al.  Detecting and Preventing Cyber Insider Threats: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[42]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[43]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[44]  Zhi-Hua Zhou,et al.  Tri-training: exploiting unlabeled data using three classifiers , 2005, IEEE Transactions on Knowledge and Data Engineering.

[45]  Qing-Long Han,et al.  Data-Driven Cyber Security in Perspective—Intelligent Traffic Analysis , 2020, IEEE Transactions on Cybernetics.

[46]  Wenjuan Li,et al.  Evaluation of Detecting Malicious Nodes Using Bayesian Model in Wireless Intrusion Detection , 2013, NSS.

[47]  Lam-for Kwok,et al.  Intrusion Detection Using Disagreement-Based Semi-supervised Learning: Detection Enhancement and False Alarm Reduction , 2012, CSS.

[48]  Wenjuan Li,et al.  Enhancing Trust Evaluation Using Intrusion Sensitivity in Collaborative Intrusion Detection Networks: Feasibility and Challenges , 2013, 2013 Ninth International Conference on Computational Intelligence and Security.

[49]  Erland Jonsson,et al.  Using active learning in intrusion detection , 2004, Proceedings. 17th IEEE Computer Security Foundations Workshop, 2004..

[50]  Saurabh Bagchi,et al.  Collaborative intrusion detection system (CIDS): a framework for accurate and efficient IDS , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[51]  Anup K. Ghosh,et al.  Detecting anomalous and unknown intrusions against programs , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[52]  Jiaqi Zheng,et al.  MAN: Mutual Attention Neural Networks Model for Aspect-Level Sentiment Classification in SIoT , 2020, IEEE Internet of Things Journal.

[53]  Salvatore J. Stolfo,et al.  A data mining framework for building intrusion detection models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[54]  Zhi-Hua Zhou,et al.  Semi-supervised learning by disagreement , 2010, Knowledge and Information Systems.

[55]  Yan Chen,et al.  Towards scalable and robust distributed intrusion alert fusion with good load balancing , 2006, LSAD '06.

[56]  Vijay Varadharajan,et al.  A Dynamic Trust Establishment and Management Framework for Wireless Sensor Networks , 2010, 2010 IEEE/IFIP International Conference on Embedded and Ubiquitous Computing.

[57]  Manas Ranjan Patra,et al.  Semi-Naïve Bayesian Method for Network Intrusion Detection System , 2009, ICONIP.

[58]  Ji Guo,et al.  A New Trust Management Framework for Detecting Malicious and Selfish Behaviour for Mobile Ad Hoc Networks , 2011, 2011IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications.

[59]  Horace Ho-Shing Ip,et al.  Enhancing collaborative intrusion detection networks against insider attacks using supervised intrusion sensitivity-based trust management model , 2017, J. Netw. Comput. Appl..

[60]  Minghua Zhang,et al.  A New Method for Filtering IDS False Positives with Semi-supervised Classification , 2012, ICIC.

[61]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.