Generative adversarial networks (GANs) promote recent successes of deep learning in fields such as computer vision and speech synthesis. However, training a GAN is notoriously tricky and unpredictable, and requires substantial efforts from both human and machines. In this paper, we introduce a novel Bayesian framework based on recent advances in deep network compression, with an attempt to mitigate the robustness issue of training GANs as well as preserving computing resources. Our novelties are twofold: (i) we leverage state-of-the-art compression techniques (e.g., hashing, pruning, vector quantization, and Huffman coding) in adversarial settings; and (ii) instead of shrinking deep nets afterwards, we adapt the network at the same time of training. The stability and efficiency of our approach are confirmed by experiments under various scenarios while the performance trade-offs are shown to be negligible. Approximation Theory and Machine Learning Conference, Purdue University, September 29-30, 2018.