Measurement-based network monitoring and inference: scalability and missing information

Using measurements collected at network monitors to infer network conditions is a promising approach for network-centric monitoring. In this context, an important question arises: given the number and locations of network monitors, how much network management resources (e.g., the number of measurements) are needed to obtain an accurate estimate of network states? We define the scalability of measurement-based network monitoring as the growth rate of the number of measurements required for accurate network monitoring/inference with respect to the size of a network. We develop a framework for investigating the scalability in the context of multicast inference with the monitors at the edges of a network. In such a framework, network monitoring/inference can be formulated as probability density estimation of network states. The growth rate is characterized through the sample complexity, which is the number of measurements needed to accurately estimate the density. The missing data framework is introduced to estimate the growth rate, where the missing data reflect unavailable measurements at the unobservable nodes without resident monitors, and the underlying nodal packet losses. We show that when the missing information is mainly due to the number of unobservable nodes, the number of measurements needed grows linearly with the size of the network, and the measurement-based inference approach is, thus, scalable. When the missing information is mainly due to the underlying nodal packet losses, the number of measurements needed grows faster than linear with the size of the network, and the measurement-based inference approach is, thus, not scalable. Our results provide guidelines for accessing feasibility of the measurement-based inference approach, and the number of probes required. We give numerical examples to illustrate some of our results.