Improving in-car emotion classification by NIR database augmentation

On-board detection of driver’s emotions has become a task of high importance for car manufacturers, as negative emotions appear to be one of the major risks for car accidents. Deep neural networks have become over the last years the state of the art methods for computer vision and image classification. Yet, their success depends upon their being trained on a comprehensive database, which should cover all of the real-life situations that may arise in practice. Most of the in-car driver monitoring cameras capture images in the near infra-red (NIR) domain therefore one needs a large database with images featuring emotions in the NIR domain. As most databases featuring human emotions contain images acquired in the visible domain, we discuss in this paper two methods of transferring the NIR-like look into the "visible" images, by using a CycleGAN style-transferring neural network trained using "paired" and "unpaired" images. We show that the resulted database augmented with NIR-like images leads to a much improved performance in emotion classification for a deep neural network, when tested on real NIR images.