A Novel Approach to Data Augmentation for Pavement Distress Segmentation

Abstract Accurate semantic segmentation ground-truths are difficult and expensive to obtain. On the other hand, the most promising approaches to automatically tackle this task, i.e. Deep Convolutional Neural Networks (CNNs), require high volumes of labeled data. We propose a new method based on deep learning for data augmentation in the context of semantic segmentation of highly-textured images. The method exploits a Generative Adversarial Network (GAN) to produce a semantic layout, then a texture synthesizer, based on a CNN, generates a new image according to the generated semantic layout and a reference real image taken from the training set. Even though our method is general and it can be utilized on a broad set of problems, we employed it on the real-world problem of detecting and localizing defects and cracks in road asphalts. We show how, starting from few labeled images, it is possible to augment small and long-tail datasets by producing new images with the associated semantic layouts. We prove the effectiveness of our approach by evaluating the performance of three different CNNs for semantic segmentation on the German Pavement Distress dataset and on a novel asphalt dataset collected by us. Results show a remarkable increase in performance, especially with low cardinality classes, when CNNs are trained on the augmented datasets with respect to original datasets.