Inferring relative popularity of internet applications by actively querying DNS caches

In this work, we propose a novel methodology that can be used to assess the relative popularity for any Internet application based on the data servers it uses. The basic idea is to infer popularity of data servers by periodically "poking" at local Domain Name servers (LDNSs) that service Domain Name System requests from a set of users running Internet applications and determining if LDNSs have cached resource records for the data servers. This approach allows us to measure the relative percentage of pokes that result in a cache hit as a coarse measure of the relative popularity of a particular data server among the users of a given LDNS. In addition, the time-to-live (TTL) of cached DNS resource records can be used to measure the gaps in time when a resource record for a data server is not cached. The cache gaps can be used to infer request interarrivals for more popular data servers.The methodology can be applied to any Internet application that uses distinguished server names and performs DNS lookups on these names as part of application use. The methodology can be used to collect usage information from any LDNS that accepts DNS queries. As example applications of the methodology, we evaluate the relative popularity of selected Web sites and the relative popularity of different Web servers serving content at a given Web site. We also apply the methodology to servers providing multimedia content, data servers for grid computing, and network game servers. We use data gathered from LDNSs of commercial and educational sites as well as Internet Service Providers serving both commercial and home customers.