Machine Learning Based CloudBot Detection Using Multi-Layer Traffic Statistics

With the rapid development of e-commerce services and online transactions, an increasing number of advanced web robots are utilized by speculators and hackers in underground economy to perform click fraud, register fake accounts and commit other kinds of frauds, seriously harming the profit of businesses and the fairness of online activities. There is solid evidence that the vast majority of such malicious bot traffic comes from data centers. The malicious bot deployed on the hosts of data centers is referred to as a CloudBot. How to detect and block CloudBots effectively has become an urgent problem in practice, while the research on it can be seldom seen in public. To this end, we propose a traffic-based quasi-real-time method for CloudBot detection using machine learning, which exploits a new sample partitioning approach, as well as innovative multi-layer features that reveal the essential difference between CloudBots and human traffic. Our method achieves 93.4% precision in the experiment and performs well on the real-world dataset, which proves to be effective to detect unknown CloudBots and combat the concept drift caused by varying time. Besides, the approach is also privacy-preserving without using any specific application layer information. We believe our work can benefit network economy security and fairness in practice.