A Survey of Deep Learning Methods for Cyber Security

This survey paper describes a literature review of deep learning (DL) methods for cyber security applications. A short tutorial-style description of each DL method is provided, including deep autoencoders, restricted Boltzmann machines, recurrent neural networks, generative adversarial networks, and several others. Then we discuss how each of the DL methods is used for security applications. We cover a broad array of attack types including malware, spam, insider threats, network intrusions, false data injection, and malicious domain names used by botnets.

[1]  Ming Zhu,et al.  End-to-end encrypted traffic classification with one-dimensional convolution neural networks , 2017, 2017 IEEE International Conference on Intelligence and Security Informatics (ISI).

[2]  Hyrum S. Anderson,et al.  Predicting Domain Generation Algorithms with Long Short-Term Memory Networks , 2016, ArXiv.

[3]  Kwangjo Kim,et al.  Improving Detection of Wi-Fi Impersonation by Fully Unsupervised Deep Learning , 2017, WISA.

[4]  Kenneth Ward Church,et al.  Margin-constrained random projections and very sparse random projections , 2006 .

[5]  Ali A. Ghorbani,et al.  Toward developing a systematic approach to generate benchmark datasets for intrusion detection , 2012, Comput. Secur..

[6]  Naveen K. Chilamkurti,et al.  Distributed attack detection scheme using deep learning approach for Internet of Things , 2017, Future Gener. Comput. Syst..

[7]  Georgios Kambourakis,et al.  Intrusion Detection in 802.11 Networks: Empirical Evaluation of Threats and a Public Dataset , 2016, IEEE Communications Surveys & Tutorials.

[8]  Xiaojiang Du,et al.  A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security , 2018, IEEE Communications Surveys & Tutorials.

[9]  Geoffrey E. Hinton,et al.  Application of Deep Belief Networks for Natural Language Understanding , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[10]  Hai Anh Tran,et al.  DGA Botnet Detection Using Supervised Learning Methods , 2017, SoICT.

[11]  Hyrum S. Anderson,et al.  DeepDGA: Adversarially-Tuned Domain Generation and Detection , 2016, AISec@CCS.

[12]  Léon Bottou,et al.  From machine learning to machine reasoning , 2011, Machine Learning.

[13]  Geoffrey E. Hinton,et al.  Training Recurrent Neural Networks , 2013 .

[14]  Andrei Petrovski,et al.  Botnet Detection in the Internet of Things using Deep Learning Approaches , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[15]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[16]  Tara N. Sainath,et al.  Deep convolutional neural networks for LVCSR , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Md Zahangir Alom,et al.  Intrusion detection using deep belief networks , 2015, 2015 National Aerospace and Electronics Conference (NAECON).

[18]  Nathan S. Netanyahu,et al.  DeepSign: Deep learning for automatic malware signature generation and classification , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[19]  Daniel Gibert Llauradó.  Convolutional neural networks for malware classification , 2016 .

[20]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[21]  Hai Anh Tran,et al.  A LSTM based framework for handling multiclass imbalance in DGA botnet detection , 2018, Neurocomputing.

[22]  Michele Colajanni,et al.  On the effectiveness of machine and deep learning for cyber security , 2018, 2018 10th International Conference on Cyber Conflict (CyCon).

[23]  Wenyi Huang,et al.  MtNet: A Multi-Task Neural Network for Dynamic Malware Classification , 2016, DIMVA.

[24]  Yoshua Bengio,et al.  Hierarchical Recurrent Neural Networks for Long-Term Dependencies , 1995, NIPS.

[25]  Ling Gao,et al.  An Intrusion Detection Model Based on Deep Belief Networks , 2014 .

[26]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[27]  Ralf C. Staudemeyer,et al.  Applying long short-term memory recurrent neural networks to intrusion detection , 2015 .

[28]  Marc'Aurelio Ranzato,et al.  Sparse Feature Learning for Deep Belief Networks , 2007, NIPS.

[29]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[30]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[31]  Konrad Rieck,et al.  DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket , 2014, NDSS.

[32]  Joshua Glasser,et al.  Bridging the Gap: A Pragmatic Approach to Generating Insider Threat Data , 2013, 2013 IEEE Security and Privacy Workshops.

[33]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[34]  Masayuki Murata,et al.  Malicious URL sequence detection using event de-noising convolutional neural network , 2017, 2017 IEEE International Conference on Communications (ICC).

[35]  Naveen K. Chilamkurti,et al.  Deep Learning: The Frontier for Distributed Attack Detection in Fog-to-Things Computing , 2018, IEEE Communications Magazine.

[36]  Jin Wei,et al.  Real-Time Detection of False Data Injection Attacks in Smart Grid: A Deep Learning-Based Intelligent Mechanism , 2017, IEEE Transactions on Smart Grid.

[37]  Hongbo Liu,et al.  Smart User Authentication through Actuation of Daily Activities Leveraging WiFi-enabled IoT , 2017, MobiHoc.

[38]  Naveen K. Chilamkurti,et al.  Leveraging LSTM Networks for Attack Detection in Fog-to-Things Communications , 2018, IEEE Communications Magazine.

[39]  Yang Gao,et al.  Apply Stacked Auto-Encoder to Spam Detection , 2015, ICSI.

[40]  Carla Purdy,et al.  Toward an Online Anomaly Intrusion Detection System Based on Deep Learning , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[41]  Yunheung Paek,et al.  LSTM-Based System-Call Language Modeling and Robust Ensemble Method for Designing Host-Based Intrusion Detection Systems , 2016, ArXiv.

[42]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[44]  Jack W. Stokes,et al.  Large-scale malware classification using random projections and neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  Mahdi Jafari Siavoshani,et al.  Deep packet: a novel approach for encrypted traffic classification using deep learning , 2017, Soft Computing.

[46]  P. J. García-Nieto,et al.  Review: machine learning techniques applied to cybersecurity , 2019, International Journal of Machine Learning and Cybernetics.

[47]  Mahmood Yousefi-Azar,et al.  Autoencoder-based feature learning for cyber security applications , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[48]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[49]  Michael Schatz,et al.  A toolkit for detecting and analyzing malicious software , 2002, 18th Annual Computer Security Applications Conference, 2002. Proceedings..

[50]  Khalid Saeed,et al.  Application of Recurrent Neural Networks for User Verification based on Keystroke Dynamics , 2016 .

[51]  Georgios Kambourakis,et al.  DDoS in the IoT: Mirai and Other Botnets , 2017, Computer.

[52]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[53]  Je-Won Kang,et al.  Intrusion Detection System Using Deep Neural Network for In-Vehicle Network Security , 2016, PloS one.

[54]  Sheng Chen,et al.  Application of Deep Belief Networks for opcode based malware detection , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[55]  Harris Drucker,et al.  Learning algorithms for classification: A comparison on handwritten digit recognition , 1995 .

[56]  Alejandro Correa Bahnsen,et al.  DeepPhish : Simulating Malicious AI , 2018 .

[57]  Aristidis Likas,et al.  Deep Belief Networks for Spam Filtering , 2007, 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007).

[58]  Brian Hutchinson,et al.  Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams , 2017, AAAI Workshops.

[59]  Félix J. García Clemente,et al.  A Self-Adaptive Deep Learning-Based System for Anomaly Detection in 5G Networks , 2018, IEEE Access.

[60]  Xiaohong Yuan,et al.  Semi-Supervised Deep Neural Network for Network Intrusion Detection , 2016 .

[61]  Razvan Benchea,et al.  Combining Restricted Boltzmann Machine and One Side Perceptron for Malware Detection , 2014, ICCS.

[62]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[63]  Xiang Zhang,et al.  Text Understanding from Scratch , 2015, ArXiv.

[64]  Mitsuaki Akiyama,et al.  Efficient Dynamic Malware Analysis Based on Network Behavior Using Deep Learning , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[65]  Qian Xu,et al.  MS-LSTM: A multi-scale LSTM model for BGP anomaly detection , 2016, 2016 IEEE 24th International Conference on Network Protocols (ICNP).

[66]  Zeng Feng,et al.  Classification for DGA-Based Malicious Domain Names with Deep Learning Architectures , 2017 .

[67]  Md Zahangir Alom,et al.  Network intrusion detection for cyber security using unsupervised deep learning approaches , 2017, 2017 IEEE National Aerospace and Electronics Conference (NAECON).

[68]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[69]  N. R. Raajan,et al.  AN INTELLECTUAL INTRUSION DETECTION SYSTEM MODEL FOR ATTACKS CLASSIFICATION USING RNN , 2017 .

[70]  Aiko Pras,et al.  An Overview of IP Flow-Based Intrusion Detection , 2010, IEEE Communications Surveys & Tutorials.

[71]  Pablo Torres,et al.  An analysis of Recurrent Neural Networks for Botnet detection behavior , 2016, 2016 IEEE Biennial Congress of Argentina (ARGENCON).

[72]  Wolfgang Banzhaf,et al.  The use of computational intelligence in intrusion detection systems: A review , 2010, Appl. Soft Comput..

[73]  Adam Doupé,et al.  Deep Android Malware Detection , 2017, CODASPY.

[74]  Razvan Pascanu,et al.  Malware classification with recurrent networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[75]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[76]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[77]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[78]  Yuancheng Li,et al.  A Hybrid Malicious Code Detection Method based on Deep Learning , 2015 .

[79]  Daniel L. Marino,et al.  Generalization of Deep Learning for Cyber-Physical System Security: A Survey , 2018, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society.

[80]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[81]  Jeffrey Pennington,et al.  Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions , 2011, EMNLP.

[82]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[83]  Martine De Cock,et al.  Inline DGA Detection with Deep Networks , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[84]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[85]  Miad Faezipour,et al.  Deep and Machine Learning Approaches for Anomaly-Based Intrusion Detection of Imbalanced Network Traffic , 2019, IEEE Sensors Letters.

[86]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[87]  Kunihiko Fukushima,et al.  Cognitron: A self-organizing multilayered neural network , 1975, Biological Cybernetics.

[88]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[89]  Kwangjo Kim,et al.  Deep Learning-based Feature Selection for Intrusion Detection System in Transport Layer 1 ) , 2016 .

[90]  Mounir Ghogho,et al.  Deep learning approach for Network Intrusion Detection in Software Defined Networking , 2016, 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM).

[91]  Yan Zhang,et al.  Deep Learning for Secure Mobile Edge Computing , 2017, ArXiv.

[92]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[93]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[94]  Ming Zhu,et al.  Malware traffic classification using convolutional neural network for representation learning , 2017, 2017 International Conference on Information Networking (ICOIN).

[95]  John Cavazos,et al.  HADM: Hybrid Analysis for Detection of Malware , 2016, IntelliSys.

[96]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[97]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[98]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[99]  Zhenlong Yuan,et al.  Droid-Sec: deep learning in android malware detection , 2015, SIGCOMM 2015.

[100]  Yang Yu,et al.  Network Intrusion Detection through Stacking Dilated Convolutional Autoencoders , 2017, Secur. Commun. Networks.

[101]  Alejandro Zunino,et al.  An empirical comparison of botnet detection methods , 2014, Comput. Secur..

[102]  Yang Yu,et al.  A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks , 2016, Sensors.

[103]  Shiven Chawla.  Deep Learning based Intrusion Detection System for Internet of Things , 2017 .

[104]  Patrick D. McDaniel,et al.  Adversarial Perturbations Against Deep Neural Networks for Malware Classification , 2016, ArXiv.

[105]  Zhenlong Yuan,et al.  DroidDetector: Android Malware Characterization and Detection Using Deep Learning , 2016 .

[106]  Eryk Dutkiewicz,et al.  Cyberattack detection in mobile cloud computing: A deep learning approach , 2017, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[107]  Xue Wang,et al.  Comparison deep learning method to traditional methods using for network intrusion detection , 2016, 2016 8th IEEE International Conference on Communication Software and Networks (ICCSN).

[108]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[109]  Pierre Lison,et al.  Automatic Detection of Malware-Generated Domains with Recurrent Neural Models , 2017, ArXiv.

[110]  Claudia Eckert,et al.  Deep Learning for Classification of Malware System Call Sequences , 2016, Australasian Conference on Artificial Intelligence.

[111]  Yajin Zhou,et al.  Dissecting Android Malware: Characterization and Evolution , 2012, 2012 IEEE Symposium on Security and Privacy.

[112]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[113]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[114]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[115]  Kurt C. Wallnau,et al.  Generating Test Data for Insider Threat Detectors , 2014, J. Wirel. Mob. Networks Ubiquitous Comput. Dependable Appl..

[116]  P. Venkata Krishna,et al.  A Deep Learning Based Artificial Neural Network Approach for Intrusion Detection , 2017, ICMC.

[117]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[118]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[119]  Yanfang Ye,et al.  DroidDelver: An Android Malware Detection System Using Deep Belief Network Based on API Call Blocks , 2016, WAIM Workshops.

[120]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[121]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[122]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[123]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[124]  Shigeki Goto,et al.  BotDetector: A robust and scalable approach toward detecting malware-infected devices , 2017, 2017 IEEE International Conference on Communications (ICC).

[125]  Yang Yu,et al.  Session-Based Network Intrusion Detection Using a Deep Learning Architecture , 2017, MDAI.

[126]  Chunhua Wang,et al.  Machine Learning and Deep Learning Methods for Cybersecurity , 2018, IEEE Access.

[127]  Jonathan A. Cox,et al.  A Signal Processing Approach for Cyber Data Classification with Deep Neural Networks , 2015, Complex Adaptive Systems.

[128]  Takeshi Yagi,et al.  Malware Detection with Deep Neural Network Using Process Behavior , 2016, 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC).

[129]  Ananthram Swami,et al.  The Limitations of Deep Learning in Adversarial Settings , 2015, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[130]  Christoph Goller,et al.  Learning task-dependent distributed representations by backpropagation through structure , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[131]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[132]  Xin Li,et al.  DeepAM: a heterogeneous deep learning framework for intelligent malware detection , 2018, Knowledge and Information Systems.

[133]  Siu-Ming Yiu,et al.  A multi-task learning model for malware classification with useful file access pattern from API call sequence , 2016, ArXiv.

[134]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[135]  Howon Kim,et al.  Applying Recurrent Neural Network to Intrusion Detection with Hessian Free Optimization , 2015, WISA.

[136]  Yanfang Ye,et al.  DL 4 MD : A Deep Learning Framework for Intelligent Malware Detection , 2016 .

[137]  Yann LeCun,et al.  Toward automatic phenotyping of developing embryos from videos , 2005, IEEE Transactions on Image Processing.

[138]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[139]  Howon Kim,et al.  Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection , 2016, 2016 International Conference on Platform Technology and Service (PlatCon).

[140]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[141]  Ishai Rosenberg,et al.  DeepOrigin: End-To-End Deep Learning For Detection Of New Malware Families , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[142]  Xavier J. A. Bellekens,et al.  Deep Learning Based Cryptographic Primitive Classification , 2017, ArXiv.

[143]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[144]  Yuval Elovici,et al.  Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection , 2018, NDSS.

[145]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[146]  Mansoor Alam,et al.  A Deep Learning Approach for Network Intrusion Detection System , 2016, EAI Endorsed Trans. Security Safety.

[147]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[148]  Konstantin Berlin,et al.  Deep neural network based malware detection using two dimensional binary program features , 2015, 2015 10th International Conference on Malicious and Unwanted Software (MALWARE).

[149]  Kenneth Ward Church,et al.  Very sparse random projections , 2006, KDD '06.

[150]  Georgia Sakellari,et al.  Cloud-Based Cyber-Physical Intrusion Detection for Vehicles Using Deep Learning , 2018, IEEE Access.

[151]  Di Wu,et al.  DeepFlow: Deep learning-based malware detection by mining Android application for abnormal usage of sensitive data , 2017, 2017 IEEE Symposium on Computers and Communications (ISCC).