NetQuest: a flexible framework for large-scale network measurement

In this paper, we present NetQuest, a flexible framework for large-scale network measurement. We apply Bayesian experimental design to select active measurements that maximize the amount of information we gain about the network path properties subject to given resource constraints. We then apply network inference techniques to reconstruct the properties of interest based on the partial, indirect observations we get through these measurements. By casting network measurement in a general Bayesian decision theoretic framework, we achieve flexibility. Our framework can support a variety of design requirements, including i) differentiated design for providing better resolution to certain parts of the network; ii) augmented design for conducting additional measurements given existing observations; and iii) joint design for supporting multiple users who are interested in different parts of the network. Our framework is also scalable and can design measurement experiments that span thousands of routers and end hosts. We develop a toolkit that realizes the framework on PlanetLab. We conduct extensive evaluation using both real traces and synthetic data. Our results show that the approach can accurately estimate network-wide and individual path properties by only monitoring within 2%-10% of paths. We also demonstrate its effectiveness in providing differentiated monitoring, supporting continuous monitoring, and satisfying the requirements of multiple users.