Selection of Auxiliary Objectives with Multi-Objective Reinforcement Learning

Efficiency of evolutionary algorithms may be increased using multi-objectivization. Multi-objectivization is performed by adding some auxiliary objectives. We consider selection of these objectives during a run of an evolutionary algorithm. One of the selection methods is based on reinforcement learning. There are several types of rewards previously used in reinforcement learning for adjusting of evolutionary algorithms. However, there is no superior reward. At the same time, reinforcement learning itself may be enhanced by multi-objectivization. So we propose a method for selection of auxiliary objectives based on multi-objective reinforcement learning, where the reward is composed of the previously used single rewards. Hence, we have double multi-objectivization: several rewards are involved in selection of several auxiliary objectives. We run the proposed method on different benchmark problems and compare it with a conventional evolutionary algorithm and a method based on single-objective reinforcement learning. Multi-objective reinforcement shows competitive behavior and is especially useful in the case when we do not know in advance which of the single rewards is efficient.