S3: a scalable sensing service for monitoring large networked systems

Efficiently operating and managing large scale distributed and federated systems is an extremely challenging problem. Current solutions are a combination of centralized management and significant over-provisioning of the infrastructure. With the explosion of new resource-intensive media applications and services, over provisioning of the infrastructure is no longer a viable option. Timely and accurate knowledge of the global environment (particularly the highly dynamic network path properties) is necessary for management of performance SLAs, just-in-time resource provisioning, near-optimal dynamic service placement and reuse, construction of network service overlays, and fast detection of failures and malicious attacks. Further, different applications require information about different aspects of the environment at different timescales. We propose S3, a Scalable Sensing Service, that achieves the above requirements and enables personalized sensing of the environment as dictated by applications.