Virtual World Bridges the Real Challenge: Automated Data Generation for Autonomous Driving

In autonomous driving research, one of the bottlenecks is the shortage of a well-annotated dataset to train deep neural networks for object detection. Specifically, a dataset focusing on harsh weather conditions is insufficient. The purpose of this research is to explore the power of utilizing synthetic data for training object detection deep neural networks under harsh weather conditions. We introduce a state-of-the-art automated pipeline to collect synthetic images from a high realism video game and generate training data which can be used for training an autonomous driving object detection neural network. We use our synthetic dataset, KITTI, and Cityscapes to train three separate object detection neural networks and employ the PASCAL object detection criteria to evaluate each neural networks' performance. The results from the experiment indicate that the neural network trained by our synthetic dataset outperforms its counterparts and achieves higher average precision (AP) in detecting images under harsh weather conditions. The result sheds a light on employing synthetic data to resolve the challenges in the real world.