Model-guided Evolution Strategies for Dynamically Balancing Exploration and Exploitation

Wide exploration of high-dimensional, multimodal design spaces is required for uncovering alternative solutions in the conceptual phase of design optimization tasks. We present a general framework for balancing exploration and exploitation during the course of the optimization that induces sequential exploitation of different optima in the search space by selecting on a solution’s fitness and a dynamic criterion termed interestingness. We use a fitness approximation model as a memory representing the parts of the search space that have been visited before. It guides the optimizer toward those areas that require additional sampling to be correctly modeled, and are hence termed interesting. Next to applying the prediction error of the model as a measure of interestingness, we consider the statistical variation in the predictions made by multiple parallel models as an alternative approach to quantify interestingness. On three artificial test functions we compare these setups running on a canonical ES to the same ES extended with either archive-based novelty, niching, or restarting, and to simply evaluating a Latin Hypercube set of sample points.