Guiding Intelligent Surveillance System by learning-by-synthesis gaze estimation

Abstract We describe a novel learning-by-synthesis method for estimating the gaze direction of an automated intelligent surveillance system. Recently, the progress of integrated learning has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distributions between the real and synthetic images, the desired performance cannot be achieved from the synthetic image learning as compared to the real images. In order to solve this problem, the previous method was to improve the authenticity of the composite image by learning the model. However, this kind of method had the disadvantage that the distortion was not improved and the level of authenticity was unstable. In order to solve this problem, we propose a new structure to improve the composite image. By referring to the idea of style transformation, we can effectively reduce the distortion of the image and minimize the need for actual data annotation. We estimate that this can produce highly realistic images, which we have demonstrated through qualitative and user research. We quantitatively evaluate the generated images by training the gaze estimation model. We use the refined synthetic dataset to show significant improvements compared with using the raw synthetic dataset.