Overload management as a fundamental service design primitive

This position paper makes the case that overload management should be a critical design goal for Internet-based systems and services. Few Internet service designs take overload into account, treating the problem as one of capacity planning rather than engineering the service to behave gracefully under extreme load. We argue that the right approach to overload management is to explicitly signal overload conditions to the application, allowing it to participate in resource management decisions. Furthermore, we claim that feedback-driven control, rather than static resource limits, should be the basis for detecting and controlling overload. We present a feedback-driven approach to overload control based on the staged event-driven architecture (SEDA) model for Internet service design. This approach makes use of adaptive admission controllers for meeting administrator-specified performance targets, such as 90th percentile response time. We demonstrate the use of these overload control mechanisms in two applications: a complex Web-based e-mail service, and a dynamic Web server benchmark.