Insights From Adversarial Fitness Functions

The performance of optimization is usually studied in specific settings where the fitness functions are highly constrained with static, stochastic or dynamic properties. This work examines what happens when the fitness function is a player engaged with the optimizer in an optimization game. Although the advantage of the fitness function is known through the No Free Lunch theorems, several deep insights about the space of possible performance measurements arise as a consequence of studying these adversarial fitness function, including: 1) Every continuous and linear method of measuring performance can be identified with the optimization game for some adversarial fitness; 2) For any convex continuous performance criterion, there is some deterministic optimizer that performs best, even when the fitness function is stochastic or dynamic; 3) Every stochastic optimization method can be viewed as a probabilistic choice over countably many deterministic methods. All of these statements hold in both finite and infinite search domains.