A Comparison of Probe-based and Router-based Methods for Measuring Packet Loss

Probe-based (active) measurements of packet loss have formed the basis for much of our empirical understanding of loss behavior in wide area networks. Routerbased (passive) loss measurements via SNMP, while not widely available, offer the potential for a more detailed perspective. In this paper we present a case study that quantitatively assesses and compares the viewpoints provided by both of these methods for measuring packet loss. Our hypothesis was that probe-based and router-based loss measurements should correlate. We investigated this by first comparing SNMP loss measurements on router interfaces to those extracted from packet traces in a series of laboratory experiments. We found these two passive measures of loss to be highly correlated. Next, we evaluated packet loss data gathered over three weeks in a widely deployed infrastructure, using both backbone router interfaces for passive measurements and co-located hosts that sent active probes in a full mesh. We found little correlation between time series of passive, router-based measures of loss and active loss probes for all of the paths in our measurement infrastructure. We also compared the distributional characteristics of these loss measurements including lengths of loss free periods, loss rates during lossy periods, and measures of loss constancy. We found the level of agreement between passive measures and active measures for each of these characteristics to be quite low. Deeper evaluation of our data indicated that current methods for active probing for packet loss suffer from high variance inherent in standard sampling techniques and from effects of end-host interface loss, which we also characterize in this work.