A Data Driven Exploration of Broadband Traffic Issues: Growth, Management, and Policy

This paper uses a novel dataset from multiple ISPs to characterize important trends in broadband traffic growth, develop a better understanding of usage-related costs, and consider the implications of this growth for both network operators (e.g. subscription plans and traffic management policies) and policy makers (e.g. interconnection policies and market structure). We begin by carefully considering the components of overall traffic growth. These include growth from additional users and devices attached to the network as well as growth from the changing behaviors of users and devices. Teasing apart these different types of growth is particularly important as markets for new subscribers saturate. Second we consider temporal properties of traffic at various time-scales which informs the tradeoffs inherent in alternative ways for defining peak traffic periods. These peak periods are important as they are primary drivers of network provisioning and service design decisions, and thus factor into cost estimation, investment planning, and policy analyses. Further, since no standard exists for defining “peak traffic periods,” the term is often employed overly causally in both technical and policy discussions. While a variety of definitions are all reasonable, understanding the differences is worthwhile. We next explore the connection between traffic, traffic during peak periods, and costs to the network providers of carrying traffic. We consider the economic, business, and policy tradeoffs for how these costs are shared. Central to this paper is a unique pool of data that a set of broadband access providers have shared with our research group. The data-set that informs this report is a large set of anonymous per-subscriber byte counts over different time intervals. Some data sets allow us to look at behavior at a 15 minute granularity. (The only per-subscriber information received was the equivalent of interface byte counters associated with anonymized subscriber identifiers so that we could track individual subscribers over multiple months. No personally identifiable information of any nature was received from providers.) This data allows us to ask and answer some key, and sometimes contentious questions, about traffic on broadband networks, such as: What is the distribution of usage across the subscriber population? What is the distribution of usage during peak periods across the subscriber population? What do daily traffic patterns look like? What is the long-term behavior of subscribers that occasionally appear in the tails of the usage distribution? How good are various heuristics (such as total traffic over a month) at approximating contributions to peak time periods? How much is per-subscriber traffic growing yearly? This rich data set provides more detailed insights than studies that are based only on long-term usage data or on aggregates of many users. Because we have per-subscriber traffic we can reconstruct the actual aggregate and peak traffic, whereas if all we had were the peak and aggregate traffic statistics (or worse only one or the other), we could only make inferences about per subscriber behavior that would depend on a number of strong simplifying assumptions. Our analysis of fat-tailed user behavior highlights just how misleading such inferences might be.