Complexity search for compressed neural networks

In this paper, we introduce a method, called Compressed Network Complexity Search (CNCS), for automatically determining the complexity of compressed networks (neural networks encoded indirectly by Fourier-type coefficients) that favors parsimonious solutions. CNCS maintains a probability distribution over complexity classes that it uses to select which class to optimize. Class probabilities are adapted based on their expected fitness, starting with a prior biased toward the simplest networks. Experiments on a challenging non-linear version of the helicopter hovering task, show that the method consistently finds simple solutions.