Methods and Systems for Understanding Large-Scale Internet Threats

Author(s): Pearce, Paul | Advisor(s): Paxson, Vern | Abstract: Large-scale Internet attacks are pervasive. A broad spectrum of actors from organized gangs of criminals to nation-states exploit the modern, layered Internet to launch politically and economically motivated attacks. The impact of these attacks is vast, ranging from billions of users experiencing Internet censorship, to tens of millions of dollars lost annually to cybercrime. Developing effective and comprehensive defenses to these large scale threats requires systematic empirical measurement.In this dissertation we develop empirical measurement methods and systems for understanding politically and economically motivated Internet threats. Specifically, we examine the problems of Internet censorship and advertising abuse in-depth and at-scale. To understand censorship, we develop Augur and Iris, methods and accompanying systems that allow us to perform global, longitudinal measurement of Internet censorship at the TCP/IP and DNS layers of the network stack—without the use of volunteers. This work addresses a range of both technical and extra-technical challenges, at a scale and fidelity not previously achieved. In combating advertising abuse, we investigate and chronicle multiple facets of the ecosystem—from clickbots to large-scale botnets to advertising injection—using a variety of empirical methods. Our work ultimately identifies fundamental structural weak-points leverageable for defense, resulting in dismantling botnets, cleaning up ad networks, and protecting users.