Robust dense light field reconstruction from sparse noisy sampling

Abstract Reconstructing densely-sampled light fields has attracted extensive attention recently. Numerous methods have been developed on the basis of various geometric characteristics of light field. However, these methods lose sight of the potential impact of input noise. Noise is ubiquitous in light field imaging of real-world scenes. The noise presented in the input views can propagate into the subsequent stages, and can significantly degrade the performance of the light field reconstruction. To remedy this problem, we propose a unified learning framework that learns the noise-invariant representations of light field, and reconstructs a clean densely-sampled light field from sparse noisy sampling. The novelty of this paper is the scheme to suppress input noise propagation and to improve the noise tolerance of view synthesis. Three convolutional sub-networks are jointly designed for this task: an encoder-decoder based unsupervised learning for the noise-independent estimation of light field depth, a residual learning based noise filtering for the unsupervised blind denoising of the sparse noisy views, and a complementary learning based rendering for the clean virtual view synthesis. We have made a comprehensive evaluation on 5 public datasets under different types of noise. The visual and quantitative comparisons demonstrate that our method not only exhibits significant superiority in noise tolerance, but also achieves excellent light field reconstruction performance.