Predict or screen your expensive assay: DoE vs. surrogates in experimental combinatorial optimization

Statistics-based Design of Experiments (DoE) methodologies are considered the gold standard in existing laboratory equipment for screening predefined experimental assays. Thus far, very little is formally known about their effectiveness in light of global optimization, particularly when compared to Evolutionary Algorithms (EAs). The current study, which was ignited by evolution-in-the-loop of functional protein expression, aims to conduct such a comparison with a focus on Combinatorial Optimization considering a dozen of decision variables, a search-space cardinality of several million combinations, under a budget of a couple of thousands of function evaluations. Due to the limited budget of evaluations, we argue that surrogate-assisted search methods become relevant for this application domain. To this end, we study a specific so-called Categorical Evolution Strategy (CatES). We systematically compare its performance, with and without being assisted by state-of-the-art surrogates, to DoE-based initialization of the search (exploiting the budget partially or entirely). Our empirical findings on noise-free benchmarks show that the surrogate-based approach is superior, as it significantly outperforms the DoE techniques and the CatES alone on the majority of the problems subject to the budget constraint. We conclude by projecting the strengths and weaknesses of EAs versus DoE, when run either directly or surrogate-aided.