DGA detection using machine learning methods

A botnet is a network of private computers infected with malicious software and controlled as a group without the knowledge of the owners. Botnets are used by cyber criminals for various malicious activities such as stealing sensitive data, sending spam, launching Distributed Denial of Service (DDoS) attacks, etc. A Command and Control (C&C) server sends commands to the compromised hosts for executing those malicious activities. In order to avoid detection, recent botnets such as Conficker, Zeus and Cryptolocker apply a technique called Domain Fluxing or Domain Name Generation Algorithms (DGA), where the infected bot is periodically generating and trying to resolve a large number of pseudorandom domain names until one of them is resolved by the DNS server. In this thesis, we survey different machine learning methods for detecting such DGAs by analyzing only the alphanumeric characteristics of the domain names in the network. We propose unsupervised models and evaluate their performance while comparing them with existing supervised models used in previous researches in this field. In addition, we propose a novel approach for unsupervised one-class SVM model for anomaly detection, which called Random One Class SVM (ROC-SVM). Our proposed unsupervised methods achieve better results than the compared supervised techniques, while detecting zero-day DGAs. If the run-time is of main concern, our novel approach for unsupervised one-class SVM is the best model among the others.