Boosting semantic segmentation with multi-task self-supervised learning for autonomous driving applications

Autonomous cars need to understand the complex and dynamic environment around them. The solutions for tasks such as classification, segmentation, and detection provide information required to understand this environment. The state-ofthe-art solutions for these tasks are based on a supervised learning, which requires a large amount of annotated data. It is extremely expensive and labour-intensive to produce such data. Unlabelled images and videos are available in large quantities at a negligible cost, but are rarely used. Self-supervised learning is a new concept that exploits unlabeled data. In this study, we use it to improve the performance of a single supervised task (i.e., semantic segmentation). We explore two selfsupervised tasks: colorization and depth prediction. The performance is assessed on two datasets: Cityscapes and KITTI. Overall, we show an improvement of up to 3 % when self-supervised tasks are trained with semantic segmentation. In conclusion, self-supervised learning improves the performance of semantic segmentation at no additional annotation nor inference-related computational costs.