Adapting a System with Noisy Outputs with Statistical Guarantees

Many complex systems are intrinsically stochastic in their behavior which complicates their control and optimization. Current self-adaptation and self-optimization approaches are not tailored to systems that have (i) complex internal behavior that is unrealistic to model explicitly, (ii) noisy outputs, (iii) high cost of bad adaptation decisions, i.e. systems that are both hard and risky to adapt at runtime. In response, we propose to model the system to be adapted as black box and apply state-of-the-art optimization techniques combined with statistical guarantees. Our main contribution is a framework that combines runtime optimization with guarantees obtained from statistical testing and with a method for handling cost of bad adaptation decisions. We evaluate the feasibility of our approach by applying it on an existing traffic navigation self-adaptation exemplar.