Towards Occupation Inference in Non-instrumented Services

Measuring the capacity and modeling the response to load of a real distributed system and its components requires painstaking instrumentation. Even though it greatly improves observability, instrumentation may not be desirable, due to cost, or possible due to legacy constraints. To model how a component responds to load and estimate its maximum capacity, and in turn act in time to preserve quality of service, we need a way to measure component occupation. Hence, recovering the occupation of internal non-instrumented components is extremely useful for system operators, as they need to ensure responsiveness of each one of these components and ways to plan resource provisioning. Unfortunately, complex systems will often exhibit non-linear responses that resist any simple closed-form decomposition. To achieve this decomposition in small subsets of non-instrumented components, we propose training a neural network that computes their respective occupations. We consider a subsystem comprised of two simple sequential components and resort to simulation, to evaluate the neural network against an optimal baseline solution. Results show that our approach can indeed infer the occupation of the layers with high accuracy, thus showing that the sampled distribution preserves enough information about the components. Hence, neural networks can improve the observability of online distributed systems in parts that lack instrumentation.