Unsupervised fisheye image correction through bidirectional loss with geometric prior

Abstract Neural network based methods for fisheye distortion correction are effective and increasingly popular, although training network require a high amount of labeled data. In this paper, we propose an unsupervised fisheye correction network to address the aforementioned issue. During the training process, the predicted parameters are employed to correct strong distortion that exists in the fisheye image and synthesize the corresponding distortion using the original distortion-free image. Thus, the network is constrained with bidirectional loss to obtain more accurate distortion parameters. We calculate the two losses at the image level as opposed to directly minimizing the difference between the predicted and ground truth of distortion parameters. Additionally, we leverage the geometric prior that the distortion distribution depends on the geometric regions of fisheye images and the straight line should be straight in the corrected images. The network focuses more on the geometric prior regions as opposed to equally perceiving the whole image without any attention mechanisms. To generate more appealing corrected results in visual appearance, we introduce a coarse-to-fine inpainting network to fill the hole regions caused by the irreversible mapping function using distortion parameters. Each module of the proposed network is differentiable, and thus the entire framework is completely end-to-end. When compared with the previous supervised methods, our method is more flexible and shows better practical applications for distortion rectification. The experiment results demonstrate that our proposed method outperforms state-of-the-art methods on the correction performance without any labeled distortion parameters.