An efficient method to detect periodic behavior in botnet traffic by analyzing control plane traffic

Botnets are large networks of bots (compromised machines) that are under the control of a small number of bot masters. They pose a significant threat to Internet’s communications and applications. A botnet relies on command and control (C2) communications channels traffic between its members for its attack execution. C2 traffic occurs prior to any attack; hence, the detection of botnet’s C2 traffic enables the detection of members of the botnet before any real harm happens. We analyze C2 traffic and find that it exhibits a periodic behavior. This is due to the pre-programmed behavior of bots that check for updates to download them every T seconds. We exploit this periodic behavior to detect C2 traffic. The detection involves evaluating the periodogram of the monitored traffic. Then applying Walker’s large sample test to the periodogram’s maximum ordinate in order to determine if it is due to a periodic component or not. If the periodogram of the monitored traffic contains a periodic component, then it is highly likely that it is due to a bot’s C2 traffic. The test looks only at aggregate control plane traffic behavior, which makes it more scalable than techniques that involve deep packet inspection (DPI) or tracking the communication flows of different hosts. We apply the test to two types of botnet, tinyP2P and IRC that are generated by SLINGbot. We verify the periodic behavior of their C2 traffic and compare it to the results we get on real traffic that is obtained from a secured enterprise network. We further study the characteristics of the test in the presence of injected HTTP background traffic and the effect of the duty cycle on the periodic behavior.

[1]  José M. F. Moura,et al.  Network traffic behavior analysis by decomposition into control and data planes , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[2]  Richard A. Davis,et al.  Time Series: Theory and Methods (2nd ed.). , 1992 .

[3]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[4]  Rajesh Krishnan,et al.  Using signal processing to analyze wireless data traffic , 2002, WiSE '02.

[5]  D. B. Preston Spectral Analysis and Time Series , 1983 .

[6]  Yiu-Tong Chan,et al.  Comparison of various periodograms for sinusoid detection and frequency estimation , 1999 .

[7]  José M. F. Moura,et al.  Detecting Botnets Using Command and Control Traffic , 2009, 2009 Eighth IEEE International Symposium on Network Computing and Applications.

[8]  Carol G. Maclennan,et al.  Study of tidal periodicities using a Transatlantic telecommunications cable , 1986 .

[9]  Guofei Gu,et al.  BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic , 2008, NDSS.

[10]  Thomas L. Marzetta,et al.  Detection, Estimation, and Modulation Theory , 1976 .

[11]  José M. F. Moura,et al.  Periodic Behavior in Botnet Command and Control Channels Traffic , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[12]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[13]  O. Yli-Harja,et al.  Robust Fisher's Test for Periodicity Detection in Noisy Biological Time Series , 2007, 2007 IEEE International Workshop on Genomic Signal Processing and Statistics.

[14]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[15]  W. Timothy Strayer,et al.  SLINGbot: A System for Live Investigation of Next Generation Botnets , 2009, 2009 Cybersecurity Applications & Technology Conference for Homeland Security.

[16]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[17]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[18]  José M. F. Moura,et al.  Long-Range Dependence Analysis of Control and Data Planes Network Traffic , 2008 .

[19]  Jason Lee,et al.  A first look at modern enterprise traffic , 2005, IMC '05.

[20]  S. Leigh,et al.  Probability and Random Processes for Electrical Engineering , 1989 .