Stabilizing Generative Adversarial Network Training: A Survey

Generative Adversarial Networks (GANs) are a type of Generative Models, which received wide attention for their ability to model complex real-world data. With a considerably large number of potential applications and a promise to alleviate some of the major issues of the field of Artificial Intelligence, recent years have seen rapid progress in the field of GANs. Despite the relative success, the process of training GANs remains challenging, suffering from instability issues such as mode collapse and vanishing gradients. This survey summarizes the approaches and methods employed for the purpose of stabilizing GAN training procedure. We provide a brief explanation behind the workings of GANs and categorize employed stabilization techniques outlining each of the approaches. Finally, we discuss the advantages and disadvantages of each of the methods, offering a comparative summary of the literature on stabilizing GAN training procedure.

[1]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[2]  Philip H. S. Torr,et al.  Multi-agent Diverse Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[4]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[5]  Jordan L. Boyd-Graber,et al.  Opponent Modeling in Deep Reinforcement Learning , 2016, ICML.

[6]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[7]  Xin Yao,et al.  Evolutionary Generative Adversarial Networks , 2018, IEEE Transactions on Evolutionary Computation.

[8]  Sepp Hochreiter,et al.  Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields , 2017, ICLR.

[9]  Karan Singh,et al.  Efficient Regret Minimization in Non-Convex Games , 2017, ICML.

[10]  Shimon Whiteson,et al.  Learning with Opponent-Learning Awareness , 2017, AAMAS.

[11]  David Berthelot,et al.  BEGAN: Boundary Equilibrium Generative Adversarial Networks , 2017, ArXiv.

[12]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[13]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[14]  Sergey Levine,et al.  A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models , 2016, ArXiv.

[15]  Christos H. Papadimitriou,et al.  Cycles in adversarial regularized learning , 2017, SODA.

[16]  Ioannis Mitliagkas,et al.  Multi-objective training of Generative Adversarial Networks with multiple discriminators , 2019, ICML.

[17]  Kenji Fukumizu,et al.  On integral probability metrics, φ-divergences and binary classification , 2009, 0901.2698.

[18]  Yann LeCun,et al.  Energy-based Generative Adversarial Networks , 2016, ICLR.

[19]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.

[20]  YoonSungroh,et al.  How Generative Adversarial Networks and Their Variants Work , 2019 .

[21]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[22]  Ole Winther,et al.  Autoencoding beyond pixels using a learned similarity metric , 2015, ICML.

[23]  Alexia Jolicoeur-Martineau,et al.  The relativistic discriminator: a key element missing from standard GAN , 2018, ICLR.

[24]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[25]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[26]  Andreas Krause,et al.  An Online Learning Approach to Generative Adversarial Networks , 2017, ICLR.

[27]  Volkan Cevher,et al.  Finding Mixed Nash Equilibria of Generative Adversarial Networks , 2018, ICML.

[28]  Andreas Krause,et al.  Evaluating GANs via Duality , 2018, ArXiv.

[29]  Bernhard Schölkopf,et al.  AdaGAN: Boosting Generative Models , 2017, NIPS.

[30]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[31]  Guo-Jun Qi,et al.  Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities , 2017, International Journal of Computer Vision.

[32]  Jacob Abernethy,et al.  On Convergence and Stability of GANs , 2018 .

[33]  Rahul Savani,et al.  Beyond Local Nash Equilibria for Adversarial Networks , 2018, BNCAI.

[34]  Matthias Bethge,et al.  A note on the evaluation of generative models , 2015, ICLR.

[35]  Chris Donahue,et al.  Synthesizing Audio with Generative Adversarial Networks , 2018, ArXiv.

[36]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[38]  Andrew M. Dai,et al.  Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step , 2017, ICLR.

[39]  Yiming Yang,et al.  MMD GAN: Towards Deeper Understanding of Moment Matching Network , 2017, NIPS.

[40]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[41]  Constantinos Daskalakis,et al.  Training GANs with Optimism , 2017, ICLR.

[42]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[43]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[44]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[45]  Shakir Mohamed,et al.  Variational Approaches for Auto-Encoding Generative Adversarial Networks , 2017, ArXiv.

[46]  A. Copeland.  Review: John von Neumann and Oskar Morgenstern, Theory of games and economic behavior , 1945 .

[47]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[48]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[49]  Alexia Jolicoeur-Martineau,et al.  GANs beyond divergence minimization , 2018, ArXiv.

[50]  John E. Hopcroft,et al.  Stacked Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Jerry Li,et al.  Towards Understanding the Dynamics of Generative Adversarial Networks , 2017, ArXiv.

[52]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[53]  Samuel A. Barnett,et al.  Convergence Problems with Generative Adversarial Networks (GANs) , 2018, ArXiv.

[54]  Mario Lucic,et al.  Are GANs Created Equal? A Large-Scale Study , 2017, NeurIPS.

[55]  Yi Zhang,et al.  Do GANs learn the distribution? Some Theory and Empirics , 2018, ICLR.

[56]  Ferenc Huszar,et al.  How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary? , 2015, ArXiv.

[57]  Edwin D. de Jong,et al.  The parallel Nash Memory for asymmetric games , 2006, GECCO.

[58]  He Ma,et al.  Generative Adversarial Parallelization , 2016, ArXiv.

[59]  Ian J. Goodfellow,et al.  On distinguishability criteria for estimating generative models , 2014, ICLR.

[60]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[61]  Fei-Yue Wang,et al.  Generative adversarial networks: introduction and outlook , 2017, IEEE/CAA Journal of Automatica Sinica.

[62]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[63]  Andrew Gordon Wilson,et al.  Bayesian GAN , 2017, NIPS.

[64]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[65]  Yingyu Liang,et al.  Generalization and Equilibrium in Generative Adversarial Nets (GANs) , 2017, ICML.

[66]  Xu Chen,et al.  Fictitious GAN: Training GANs with Historical Models , 2018, ECCV.

[67]  Trung Le,et al.  MGAN: Training Generative Adversarial Nets with Multiple Generators , 2018, ICLR.

[68]  Jascha Sohl-Dickstein,et al.  Improved generator objectives for GANs , 2016, ArXiv.

[69]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[70]  Florian Schäfer,et al.  Competitive Gradient Descent , 2019, NeurIPS.

[71]  Trung Le,et al.  Dual Discriminator Generative Adversarial Nets , 2017, NIPS.

[72]  Vishnu Naresh Boddeti,et al.  Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking , 2017, ArXiv.

[73]  Saifuddin Hitawala,et al.  Comparative Study on Generative Adversarial Networks , 2018, ArXiv.

[74]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[75]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[76]  Sebastian Nowozin,et al.  Which Training Methods for GANs do actually Converge? , 2018, ICML.

[77]  Gunnar Rätsch,et al.  Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs , 2017, ArXiv.

[78]  Peter Stone,et al.  Autonomous agents modelling other agents: A comprehensive survey and open problems , 2017, Artif. Intell..

[79]  Marc G. Bellemare,et al.  The Cramer Distance as a Solution to Biased Wasserstein Gradients , 2017, ArXiv.

[80]  Vinay P. Namboodiri,et al.  Message Passing Multi-Agent GANs , 2016, ArXiv.

[81]  Tomas E. Ward,et al.  Generative Adversarial Networks: A Survey and Taxonomy , 2019, ArXiv.

[82]  Léon Bottou,et al.  Towards Principled Methods for Training Generative Adversarial Networks , 2017, ICLR.

[83]  David Pfau,et al.  Connecting Generative Adversarial Networks and Actor-Critic Methods , 2016, ArXiv.

[84]  Bo An,et al.  Stackelberg Security Games: Looking Beyond a Decade of Success , 2018, IJCAI.