Darknet Traffic Big-Data Analysis and Network Management to Real-Time Automating the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real time. Based on a weight agnostic neural networks (WANNs) methodology, we propose an automated searching neural net architecture strategy that can perform various tasks such as identifying zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

[1]  Konstantinos Demertzis,et al.  Evolving Computational Intelligence System for Malware Detection , 2014, CAiSE Workshops.

[2]  Siromani Duddu,et al.  Secure Socket Layer Stripping Attack Using Address Resolution Protocol Spoofing , 2020, 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS).

[3]  Heike Trautmann,et al.  Automated Algorithm Selection: Survey and Perspectives , 2018, Evolutionary Computation.

[4]  Brandon M. Greenwell,et al.  Interpretable Machine Learning , 2019, Hands-On Machine Learning with R.

[5]  Aidong Men,et al.  A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data , 2017, Comput. Intell. Neurosci..

[6]  Zuofeng Gao,et al.  The Shapley value of convex compound stochastic cooperative game , 2011, 2011 2nd International Conference on Artificial Intelligence, Management Science and Electronic Commerce (AIMSEC).

[7]  Mingzhe Li,et al.  Classification of Illegal Activities on the Dark Web , 2019, Proceedings of the 2019 2nd International Conference on Information Science and Systems.

[8]  Konstantinos Demertzis,et al.  Anomaly detection via blockchained deep learning smart contracts in industry 4.0 , 2020, Neural Computing and Applications.

[9]  Deepak Gupta,et al.  An Effective Training Scheme for Deep Neural Network in Edge Computing Enabled Internet of Medical Things (IoMT) Systems , 2020, IEEE Access.

[10]  Lei Cao,et al.  Learning to Learn: Hierarchical Meta-Critic Networks , 2019, IEEE Access.

[11]  Jitender Kumar,et al.  Botnet and P2P Botnet Detection Strategies: A Review , 2018, 2018 International Conference on Communication and Signal Processing (ICCSP).

[12]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[13]  Zhouguo Chen,et al.  Big Data Analyses of ZeroNet Sites for Exploring the New Generation DarkWeb , 2020, ICSIM.

[14]  Nasir Ghani,et al.  A Machine Learning Model for Classifying Unsolicited IoT Devices by Observing Network Telescopes , 2018, 2018 14th International Wireless Communications & Mobile Computing Conference (IWCMC).

[15]  Susan Stepney,et al.  Evolving Carbon Nanotube Reservoir Computers , 2016, UCNC.

[16]  Hans-J. Briegel,et al.  Meta-learning within Projective Simulation , 2016, IEEE Access.

[17]  Konstantinos Demertzis,et al.  GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification , 2020, Algorithms.

[18]  Meihong Yang,et al.  Dark web forum correlation analysis research , 2019, 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC).

[19]  Cedric Baudoin,et al.  Towards the Deployment of Machine Learning Solutions in Network Traffic Classification: A Systematic Survey , 2019, IEEE Communications Surveys & Tutorials.

[20]  Xue Liu,et al.  Shapley Homology: Topological Analysis of Sample Influence for Neural Networks , 2019, Neural Computation.

[21]  Peter L. Bartlett,et al.  Efficient agnostic learning of neural networks with bounded fan-in , 1996, IEEE Trans. Inf. Theory.

[22]  Konstantinos Demertzis,et al.  Blockchain-based Consents Management for Personal Data Processing in the IoT Ecosystem , 2018, ICETE.

[23]  Changhe Yuan,et al.  Diversity in Neural Architecture Search , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[24]  Arash Habibi Lashkari,et al.  DIDarknet: A Contemporary Approach to Detect and Characterize the Darknet Traffic using Deep Image Learning , 2020, ICCNS.

[25]  Konstantinos Demertzis,et al.  ADvoCATE: A Consent Management Platform for Personal Data Processing in the IoT Using Blockchain Technology , 2018, SecITC.

[26]  Konstantinos Demertzis,et al.  Gryphon: a semi-supervised anomaly detection system based on one-class evolving spiking neural network , 2019, Neural Computing and Applications.

[27]  Mohammad Zulkernine,et al.  Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection , 2006, 2006 IEEE International Conference on Communications.

[28]  Jim Austin,et al.  A neural architecture for fast rule matching , 1995, Proceedings 1995 Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems.

[29]  Stefanos Gritzalis,et al.  Trust establishment in sensor networks: behaviour-based, certificate-based and a combinational approach , 2008, Int. J. Syst. Syst. Eng..

[30]  Zhen Ling,et al.  TorWard: Discovery of malicious traffic over Tor , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[31]  Shikha Agrawal,et al.  A Survey on Feature Selection Techniques for Internet Traffic Classification , 2015, 2015 International Conference on Computational Intelligence and Communication Networks (CICN).

[32]  V.A. Skormin,et al.  Anomalous packet identification for network intrusion detection , 2004, Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004..

[33]  Paulo Shakarian,et al.  Predicting Hacker Adoption on Darkweb Forums Using Sequential Rule Mining , 2018, 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom).

[34]  Seokjoo Shin,et al.  Deanonymizing schemes of hidden services in tor network: A survey , 2015, 2015 International Conference on Information Networking (ICOIN).

[35]  Ali Dehghantanha,et al.  A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting , 2018, Future Gener. Comput. Syst..

[36]  Paulo Shakarian,et al.  Predicting Cyber Threats through Hacker Social Networks in Darkweb and Deepweb Forums , 2017 .

[37]  Hsiao-Hwa Chen,et al.  Trust and Reputation Systems for Wireless Sensor Networks , 2009 .

[38]  Konstantinos Demertzis,et al.  A Dynamic Ensemble Learning Framework for Data Stream Analysis and Real-Time Threat Detection , 2018, ICANN.

[39]  Qingquan Song,et al.  Auto-Keras: An Efficient Neural Architecture Search System , 2018, KDD.

[40]  Ramzi A. Haraty,et al.  The TOR data communication system: A survey , 2014, 2014 IEEE Symposium on Computers and Communications (ISCC).

[41]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[42]  Konstantinos Demertzis,et al.  Cyber Threats to Industrial IoT: A Survey on Attacks and Countermeasures , 2021 .

[43]  Konstantinos Demertzis,et al.  A Hybrid Network Anomaly and Intrusion Detection Approach Based on Evolving Spiking Neural Network Classification , 2013, e-Democracy.

[44]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[45]  Joachim Fabini,et al.  Botnet Communication Patterns , 2017, IEEE Communications Surveys & Tutorials.

[46]  Mohsen Guizani,et al.  Traffic Analysis Attacks on Tor: A Survey , 2020, 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT).

[47]  Ming Yang,et al.  Hadoop-based Dark Web Threat Intelligence Analysis Framework , 2019, 2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC).

[48]  Lily Rachmawati,et al.  A theoretical study of the relationship between an ELM network and its subnetworks , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[49]  Konstantinos Demertzis,et al.  Extreme deep learning in biosecurity: the case of machine hearing for marine species identification , 2018, J. Inf. Telecommun..

[50]  S. Ramakrishnan,et al.  PERFORMANCE COMPARISON FOR INTRUSION DETECTION SYSTEM USING NEURAL NETWORK WITH KDD DATASET , 2014, SOCO 2014.

[51]  Ece Guran Schmidt,et al.  Machine learning algorithms for accurate flow-based network traffic classification: Evaluation and comparison , 2010, Perform. Evaluation.

[52]  M. Venkatesan,et al.  Hybrid Approach for Intrusion Detection System , 2018, 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT).

[53]  Domenico Ciuonzo,et al.  A Dive into the Dark Web: Hierarchical Traffic Classification of Anonymity Tools , 2020, IEEE Transactions on Network Science and Engineering.

[54]  Douglas Salane The Internet (Deep and Dark Web) , 2019 .

[55]  Jason R. Marden,et al.  Design Tradeoffs in Concave Cost-Sharing Games , 2018, IEEE Transactions on Automatic Control.

[56]  Sherif Sakr,et al.  A Decision Support Framework for AutoML Systems: A Meta-Learning Approach , 2019, 2019 International Conference on Data Mining Workshops (ICDMW).

[57]  Valentina Markova,et al.  Analysis and Categorization of Traffic Streams by Artificial Intelligence , 2019, 2019 International Conference on Biomedical Innovations and Applications (BIA).

[58]  Kang Song,et al.  Multi layer multi objective extreme learning machine , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[59]  Bowen Yang,et al.  Research on Network Traffic Identification based on Machine Learning and Deep Packet Inspection , 2019, 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[60]  Paulo Shakarian,et al.  Finding Cryptocurrency Attack Indicators Using Temporal Logic and Darkweb Data , 2018, 2018 IEEE International Conference on Intelligence and Security Informatics (ISI).

[61]  Huaqun Guo,et al.  A Survey on IIoT Security , 2019, 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS).

[62]  Herbert Jaeger,et al.  Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks , 2013, Neural Computation.

[63]  S. P. Shantharajah,et al.  A Study on NSL-KDD Dataset for Intrusion Detection System Based on Classification Algorithms , 2015 .

[64]  D Vasumathi,et al.  Review on anomaly based network intrusion detection system , 2017, 2017 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT).

[65]  Mounir Ghogho,et al.  Analysis of Hacking Related Trade in the Darkweb , 2018, 2018 IEEE International Conference on Intelligence and Security Informatics (ISI).

[66]  Julien Sylvestre,et al.  Computing with networks of nonlinear mechanical oscillators , 2017, PloS one.

[67]  Dongil Han,et al.  A Novel Encoding Scheme for Complex Neural Architecture Search , 2019, 2019 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC).

[68]  Jinoh Kim,et al.  A survey of deep learning-based network anomaly detection , 2017, Cluster Computing.

[69]  Shisheng Zhong,et al.  A novel anomaly detection method for gas turbines using weight agnostic neural network search , 2020, 2020 Asia-Pacific International Symposium on Advanced Reliability and Maintenance Modeling (APARM).

[70]  Amol Borkar,et al.  A survey on Intrusion Detection System (IDS) and Internal Intrusion Detection and protection system (IIDPS) , 2017, 2017 International Conference on Inventive Computing and Informatics (ICICI).

[71]  Peter Michael Young,et al.  A tighter bound for the echo state property , 2006, IEEE Transactions on Neural Networks.

[72]  Antonella Santone,et al.  Real-Time SCADA Attack Detection by Means of Formal Methods , 2019, 2019 IEEE 28th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE).

[73]  Evizal Abdul Kadir,et al.  Network Traffic Monitoring and Analysis Using Packet Sniffer , 2019, 2019 International Conference on Advanced Communication Technologies and Networking (CommNet).

[74]  Florin Leon,et al.  Optimizing neural network topology using Shapley value , 2014, 2014 18th International Conference on System Theory, Control and Computing (ICSTCC).

[75]  Christos Makris,et al.  Model-Agnostic Interpretability with Shapley Values , 2019, 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA).

[76]  Paulo Shakarian,et al.  Community Finding of Malware and Exploit Vendors on Darkweb Marketplaces , 2018, 2018 1st International Conference on Data Intelligence and Security (ICDIS).

[77]  Konstantinos Demertzis,et al.  Evolving Smart URL Filter in a Zone-Based Policy Firewall for Detecting Algorithmically Generated Malicious Domains , 2015, SLDS.

[78]  George Athanasopoulos,et al.  Forecasting: principles and practice , 2013 .

[79]  C. Căleanu,et al.  Deep Learning based Eye Gaze Tracking for Automotive Applications: An Auto-Keras Approach , 2020, 2020 International Symposium on Electronics and Telecommunications (ISETC).

[80]  Konstantinos Demertzis,et al.  Identifying data streams anomalies by evolving spiking restricted Boltzmann machines , 2019, Neural Computing and Applications.

[81]  Ren Ping Liu,et al.  ResInNet: A Novel Deep Neural Network With Feature Reuse for Internet of Things , 2019, IEEE Internet of Things Journal.