Realistic view synthesis of a structured traffic environment via adversarial training

Generating a realistic image from a novel viewpoint has always been a key problem in image-based rendering and other related domains. In this paper we utilize the state-of-the-art generative adversarial networks(GAN) to synthesize novel views of a structured scene. Based on our proposed representations for traffic scene, a realistic image of a certain viewpoint can be generated via conditional GANs, given the geometric layout of the corresponding position and pose. In order to preserve the geometric property of the input in the generated image, we propose a simple but effective constraint in the generator network. Qualitative and comparative results have validated our method as well as shown its effectiveness and efficiency.

[1]  Bastian Goldlücke,et al.  Bayesian View Synthesis and Image-Based Rendering Principles , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Minh N. Do,et al.  Probability-Based Rendering for View Synthesis , 2014, IEEE Transactions on Image Processing.

[3]  Li Li,et al.  Intelligence Testing for Autonomous Vehicles: A New Approach , 2016, IEEE Transactions on Intelligent Vehicles.

[4]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[5]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[6]  Chi Zhang,et al.  RoadView: A traffic scene simulator for autonomous vehicle simulation testing , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[7]  Abhinav Gupta,et al.  Generative Image Modeling Using Style and Structure Adversarial Networks , 2016, ECCV.

[8]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[9]  Nanning Zheng,et al.  Three-Dimensional Traffic Scenes Simulation From Road Image Sequences , 2016, IEEE Transactions on Intelligent Transportation Systems.

[10]  Wojciech Matusik,et al.  CG2Real: Improving the Realism of Computer Generated Images Using a Large Collection of Photographs , 2011, IEEE Transactions on Visualization and Computer Graphics.

[11]  Jitendra Malik,et al.  View Synthesis by Appearance Flow , 2016, ECCV.

[12]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.