Background and foreground disentangled generative adversarial network for scene image synthesis

Abstract Despite recent generative models have made remarkable progress on adversarial image synthesis, it is still a pivotal and frontier problem to generate high-fidelity images containing diverse entities and complex scene layouts from structured descriptions. To this end, we present a Background and Foreground Disentangled Generative Adversarial Network (BFD-GAN) to synthesize high-quality images from scene graphs. First, our method uses the graph convolutional network to infer a semantic background from the input scene graph. Then, the foreground parsing module that encourages unsupervised generation, is proposed to calculate semantically related foregrounds with fine-grained geometric properties. Furthermore, we also employ the foreground-background integrating module for the final image generation, during which the foreground-relation aware attention is introduced to refine and fuse the inferred foregrounds into the background. Evaluated on the COCO-Stuff and Visual Genome datasets, we benchmark our model against existing methods and show that our BFD-GAN is more capable of generating complex backgrounds and corresponding sharp foregrounds with given scene structures.