Maximal Correlation: An Alternative Criterion for Training Generative Networks

Generative adversarial networks (GANs) require the design of a generator and a discriminator network which is achieved by solving min-max optimization problems. Min-max/adversarial optimizations are implemented with the help of two stochastic gradient algorithms, one for each optimization problems. This data-driven approach is known to suffer from non-robustness and need for excessive computations and processing time. In this work we propose a kernel based correlation criterion which we only maximize. Under ideal conditions this non-adversarial approach is shown to achieve the same goal as the existing adversarial methods. Under a pure datadriven scenario we only need a generator network which we train with a single gradient algorithm. Since the proposed criterion is a nonlinear combination of three expectations of functions, as opposed to the standard case of a single expectation of a function, deriving a gradient algorithm that optimizes it, is not straightforward. The solution we developed for a general optimization problem involving nonlinear functions of expectations, clearly constitutes an additional interesting result.