The Next Generation Cognitive Security Operations Center: Adaptive Analytic Lambda Architecture for Efficient Defense against Adversarial Attacks

A Security Operations Center (SOC) is a central technical level unit responsible for monitoring, analyzing, assessing, and defending an organization’s security posture on an ongoing basis. The SOC staff works closely with incident response teams, security analysts, network engineers and organization managers using sophisticated data processing technologies such as security analytics, threat intelligence, and asset criticality to ensure security issues are detected, analyzed and finally addressed quickly. Those techniques are part of a reactive security strategy because they rely on the human factor, experience and the judgment of security experts, using supplementary technology to evaluate the risk impact and minimize the attack surface. This study suggests an active security strategy that adopts a vigorous method including ingenuity, data analysis, processing and decision-making support to face various cyber hazards. Specifically, the paper introduces a novel intelligence driven cognitive computing SOC that is based exclusively on progressive fully automatic procedures. The proposed λ-Architecture Network Flow Forensics Framework (λ-ΝF3) is an efficient cybersecurity defense framework against adversarial attacks. It implements the Lambda machine learning architecture that can analyze a mixture of batch and streaming data, using two accurate novel computational intelligence algorithms. Specifically, it uses an Extreme Learning Machine neural network with Gaussian Radial Basis Function kernel (ELM/GRBFk) for the batch data analysis and a Self-Adjusting Memory k-Nearest Neighbors classifier (SAM/k-NN) to examine patterns from real-time streams. It is a forensics tool for big data that can enhance the automate defense strategies of SOCs to effectively respond to the threats their environments face.

[1]  Xiangliang Zhang,et al.  Network traffic monitoring, analysis and anomaly detection [Guest Editorial] , 2011, IEEE Netw..

[2]  Konstantinos Demertzis,et al.  Extreme deep learning in biosecurity: the case of machine hearing for marine species identification , 2018, J. Inf. Telecommun..

[3]  Wenke Lee,et al.  Polymorphic Blending Attacks , 2006, USENIX Security Symposium.

[4]  Richard E. Harang,et al.  Rapid Permissions-Based Detection and Analysis of Mobile Malware Using Random Decision Forests , 2013, MILCOM 2013 - 2013 IEEE Military Communications Conference.

[5]  L. Iliadis,et al.  Ladon: A Cyber-Threat Bio-Inspired Intelligence Management System , 2016 .

[6]  Konstantinos Demertzis,et al.  Evolving Smart URL Filter in a Zone-Based Policy Firewall for Detecting Algorithmically Generated Malicious Domains , 2015, SLDS.

[7]  Patrick Haffner,et al.  ACAS: automated construction of application signatures , 2005, MineNet '05.

[8]  Kang G. Shin,et al.  Detecting SYN flooding attacks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[9]  Fabrício Enembreck,et al.  An investigation of the hoeffding adaptive tree for the problem of network intrusion detection , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[10]  Konstantinos Demertzis,et al.  HISYCOL a hybrid computational intelligence system for combined machine learning: the case of air pollution modeling in Athens , 2015, Neural Computing and Applications.

[11]  Konstantinos Demertzis,et al.  MOLESTRA : A MultiTask Learning Approach for Real-Time Big Data Analytics , 2018 .

[12]  Inder Monga,et al.  Lambda architecture for cost-effective batch and speed big data processing , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[13]  Simin Nadjm-Tehrani,et al.  Crowdroid: behavior-based malware detection system for Android , 2011, SPSM '11.

[14]  Konstantinos Demertzis,et al.  Commentary: Aedes albopictus and Aedes japonicas—two invasive mosquito species with different temperature niches in Europe , 2017, Front. Environ. Sci..

[15]  Konstantinos Demertzis,et al.  Hybrid Unsupervised Modeling of Air Pollution Impact to Cardiovascular and Respiratory Diseases , 2017, Int. J. Inf. Syst. Crisis Response Manag..

[16]  Blaine Nelson,et al.  Exploiting Machine Learning to Subvert Your Spam Filter , 2008, LEET.

[17]  Yanfang Ye,et al.  Adversarial Machine Learning in Malware Detection: Arms Race between Evasion Attack and Defense , 2017, 2017 European Intelligence and Security Informatics Conference (EISIC).

[18]  Konstantinos Demertzis,et al.  Hybrid Soft Computing for Atmospheric Pollution-Climate Change Data Mining , 2018, Trans. Comput. Collect. Intell..

[19]  Konstantinos Demertzis,et al.  Fuzzy Cognitive Maps for Long-Term Prognosis of the Evolution of Atmospheric Pollution, Based on Climate Change Scenarios: The Case of Athens , 2016, ICCCI.

[20]  Yoji Yamato,et al.  Proposal of Lambda Architecture Adoption for Real Time Predictive Maintenance , 2016, 2016 Fourth International Symposium on Computing and Networking (CANDAR).

[21]  Konstantinos Demertzis,et al.  Machine learning use in predicting interior spruce wood density utilizing progeny test information , 2017, Neural Computing and Applications.

[22]  Konstantinos Demertzis,et al.  Bio-inspired Hybrid Intelligent Method for Detecting Android Malware , 2016, KICSS.

[23]  James Newsome,et al.  Paragraph: Thwarting Signature Learning by Training Maliciously , 2006, RAID.

[24]  Fabio Roli,et al.  Machine Learning Methods for Computer Security (Dagstuhl Perspectives Workshop 12371) , 2012, Dagstuhl Reports.

[25]  João Gama,et al.  Evaluation of recommender systems in streaming environments , 2015, ArXiv.

[26]  Jun Zhang,et al.  An Effective Network Traffic Classification Method with Unknown Flow Detection , 2013, IEEE Transactions on Network and Service Management.

[27]  Kevin Aretz,et al.  Asymmetric Loss Functions and the Rationality of Expected Stock Returns , 2009 .

[28]  Felix C. Freiling,et al.  Measuring and Detecting Fast-Flux Service Networks , 2008, NDSS.

[29]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[30]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[31]  Zahir Tari,et al.  A Dynamic Resource Controller for a Lambda Architecture , 2017, 2017 46th International Conference on Parallel Processing (ICPP).

[32]  Konstantinos Demertzis,et al.  Blockchain-based Consents Management for Personal Data Processing in the IoT Ecosystem , 2018, ICETE.

[33]  Yuval Elovici,et al.  Automated Static Code Analysis for Classifying Android Applications Using Machine Learning , 2010, 2010 International Conference on Computational Intelligence and Security.

[34]  Victor C. M. Leung,et al.  Extreme Learning Machines [Trends & Controversies] , 2013, IEEE Intelligent Systems.

[35]  Konstantinos Demertzis,et al.  A Spiking One-Class Anomaly Detection Framework for Cyber-Security on Industrial Control Systems , 2017, EANN.

[36]  Konstantinos Demertzis,et al.  Fast and low cost prediction of extreme air pollution values with hybrid unsupervised learning , 2016, Integr. Comput. Aided Eng..

[37]  Guang-Bin Huang,et al.  What are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle , 2015, Cognitive Computation.

[38]  Konstantinos Demertzis,et al.  Evolving Computational Intelligence System for Malware Detection , 2014, CAiSE Workshops.

[39]  Konstantinos Demertzis,et al.  Comparative analysis of exhaust emissions caused by chainsaws with soft computing and statistical approaches , 2018, International Journal of Environmental Science and Technology.

[40]  Konstantinos Demertzis,et al.  The Next Generation Cognitive Security Operations Center: Network Flow Forensics Using Cybersecurity Intelligence , 2018, Big Data Cogn. Comput..

[41]  Wim Mees,et al.  A comparative analysis of visualisation techniques to achieve cyber situational awareness in the military , 2018, 2018 International Conference on Military Communications and Information Systems (ICMCIS).

[42]  Riccardo Scandariato,et al.  Predicting vulnerable classes in an Android application , 2012, MetriSec '12.

[43]  Chun-Ying Huang,et al.  Fast-Flux Bot Detection in Real Time , 2010, RAID.

[44]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Jalal Omer Atoum,et al.  A Model for Detecting Tor Encrypted Traffic using Supervised Machine Learning , 2015 .

[46]  Konstantinos Demertzis,et al.  Classifying with fuzzy chi-square test: The case of invasive species , 2018 .

[47]  Daniel Massey,et al.  Detecting encrypted botnet traffic , 2013, INFOCOM Workshops.

[48]  Pedro M. Domingos,et al.  Adversarial classification , 2004, KDD.

[49]  Konstantinos Demertzis,et al.  A Computational Intelligence System Identifying Cyber-Attacks on Smart Energy Grids , 2018 .

[50]  David A. Wagner,et al.  Analyzing inter-application communication in Android , 2011, MobiSys '11.

[51]  Konstantinos Demertzis,et al.  Detecting invasive species with a bio-inspired semi-supervised neurocomputing approach: the case of Lagocephalus sceleratus , 2017, Neural Computing and Applications.

[52]  Konstantinos Demertzis,et al.  A Dynamic Ensemble Learning Framework for Data Stream Analysis and Real-Time Threat Detection , 2018, ICANN.

[53]  Venu Govindaraju,et al.  Robustness of multimodal biometric fusion methods against spoof attacks , 2009, J. Vis. Lang. Comput..

[54]  Siu-Ming Yiu,et al.  A Survey on Regular Expression Matching for Deep Packet Inspection: Applications, Algorithms, and Hardware Platforms , 2016, IEEE Communications Surveys & Tutorials.

[55]  João Pedro Hespanha,et al.  Observability of linear systems under adversarial attacks , 2015, 2015 American Control Conference (ACC).

[56]  Konstantinos Demertzis,et al.  A Bio-Inspired Hybrid Artificial Intelligence Framework for Cyber Security , 2015 .

[57]  Konstantinos Demertzis,et al.  An innovative soft computing system for smart energy grids cybersecurity , 2018 .

[58]  Konstantinos Demertzis,et al.  Hybrid intelligent modeling of wild fires risk , 2018, Evol. Syst..

[59]  Konstantinos Demertzis,et al.  FuSSFFra, a fuzzy semi-supervised forecasting framework: the case of the air pollution in Athens , 2018, Neural Computing and Applications.

[60]  Konstantinos Demertzis,et al.  Artificial Intelligence Applications and Innovations: 18th IFIP WG 12.5 International Conference, AIAI 2022, Hersonissos, Crete, Greece, June 17–20, 2022, Proceedings, Part II , 2022, IFIP Advances in Information and Communication Technology.

[61]  Konstantinos Demertzis,et al.  SAME: An Intelligent Anti-malware Extension for Android ART Virtual Machine , 2015, ICCCI.

[62]  Shishir Nagaraja,et al.  On the Reliability of Network Measurement Techniques Used for Malware Traffic Analysis , 2014, Security Protocols Workshop.

[63]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[64]  Geoff Holmes,et al.  Evaluation methods and decision theory for classification of streaming data with temporal dependence , 2015, Machine Learning.

[65]  Konstantinos Demertzis,et al.  Intelligent Bio-Inspired Detection of Food Borne Pathogen by DNA Barcodes: The Case of Invasive Fish Species Lagocephalus Sceleratus , 2015, EANN.

[66]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[67]  Konstantinos Demertzis,et al.  A Hybrid Network Anomaly and Intrusion Detection Approach Based on Evolving Spiking Neural Network Classification , 2013, e-Democracy.

[68]  Konstantinos Demertzis,et al.  Adaptive Elitist Differential Evolution Extreme Learning Machines on Big Data: Intelligent Recognition of Invasive Species , 2016, INNS Conference on Big Data.

[69]  A. Nur Zincir-Heywood,et al.  A Comparison of three machine learning techniques for encrypted network traffic analysis , 2011, 2011 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA).

[70]  Heiko Wersing,et al.  KNN Classifier with Self Adjusting Memory for Heterogeneous Concept Drift , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[71]  A. Khan,et al.  Optimised lambda architecture for monitoring WLCG using spark and spark streaming , 2016, 2016 IEEE Nuclear Science Symposium, Medical Imaging Conference and Room-Temperature Semiconductor Detector Workshop (NSS/MIC/RTSD).

[72]  Clare Stanier,et al.  Towards Differentiating Business Intelligence, Big Data, Data Analytics and Knowledge Discovery , 2016, ERP Future.

[73]  Konstantinos Demertzis,et al.  Temporal Modeling of Invasive Species' Migration in Greece from Neighboring Countries Using Fuzzy Cognitive Maps , 2018, AIAI.

[74]  Konstantinos Demertzis,et al.  Computational intelligence anti-malware framework for android OS , 2017, Vietnam Journal of Computer Science.

[75]  Emilio Frazzoli,et al.  Robust and resilient estimation for Cyber-Physical Systems under adversarial attacks , 2016, 2016 American Control Conference (ACC).