Real-time botnet detection using nonnegative tucker decomposition

This study focuses on darknet traffic analysis and applies tensor factorization in order to detect coordinated group activities, such as a botnet. Tensor factorization is a powerful tool for extracting co-occurrence patterns that is highly interpretable and can handle more variables than matrix factorization. We propose a simple method for detecting group activities from its extracted features. However, tensor factorization requires too high a computational cost to run in real time. To address this problem, we implemented a two-step algorithm in order to achieve fast, memory-efficient factorization. We also utilize nonnegative Tucker decomposition, one of the tensor factorization methods, because it has non-negativity constraints, to avoid physically unreasonable results. Finally, we introduce our prototype implementation of the proposed scheme, with which we demonstrate the effectiveness of the scheme by reviewing several past security incidents.

[1]  Jun'ichi Takeuchi,et al.  Botnet Detection Based on Non-negative Matrix Factorization and the MDL Principle , 2012, ICONIP.

[2]  Hisashi Kashima,et al.  Eigenspace-based anomaly detection in computer systems , 2004, KDD.

[3]  Prateek Jain,et al.  Tensor vs. Matrix Methods: Robust Tensor Decomposition under Block Sparse Perturbations , 2015, AISTATS.

[4]  Mourad Debbabi,et al.  Fingerprinting Internet DNS Amplification DDoS Activities , 2014, 2014 6th International Conference on New Technologies, Mobility and Security (NTMS).

[5]  Seungjin Choi,et al.  Nonnegative Tucker Decomposition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Yi Zhou,et al.  Understanding the Mirai Botnet , 2017, USENIX Security Symposium.

[7]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[8]  A. Cichocki,et al.  Generalizing the column–row matrix decomposition to multi-way arrays , 2010 .

[9]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[10]  Michael Bailey,et al.  Taming the 800 Pound Gorilla: The Rise and Decline of NTP DDoS Attacks , 2014, Internet Measurement Conference.

[11]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[12]  Tao Ban,et al.  Detection of DDoS Backscatter Based on Traffic Features of Darknet TCP Packets , 2014, 2014 Ninth Asia Joint Conference on Information Security.

[13]  Tonio Ball,et al.  Multilinear Subspace Regression: An Orthogonal Tensor Decomposition Approach , 2011, NIPS.

[14]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[15]  D. Inoue,et al.  nicter: An Incident Analysis System Toward Binding Network Monitoring with Malware Analysis , 2008, 2008 WOMBAT Workshop on Information Security Threats Data Collection and Sharing.

[16]  Hiroshi Sawada,et al.  Brand-Choice Analysis using Non-negative Tensor Factorization , 2015 .

[17]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[18]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[19]  Andrzej Cichocki,et al.  Efficient Nonnegative Tucker Decompositions: Algorithms and Uniqueness , 2014, IEEE Transactions on Image Processing.

[20]  Mitsuaki Akiyama,et al.  A Proposal of Metrics for Botnet Detection Based on Its Cooperative Behavior , 2007, 2007 International Symposium on Applications and the Internet Workshops.

[21]  Tao Ban,et al.  A neural network model for detecting DDoS attacks using darknet traffic features , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[22]  Nobuyuki Otsu,et al.  ATlreshold Selection Method fromGray-Level Histograms , 1979 .