Dual attention autoencoder for all-weather outdoor lighting estimation

Accurate outdoor illumination estimation is not easy due to extremely complicated sky appearance and the mutual interference of the sun and sky. In view of these challenges, we present a new deep approach for the estimation of all-weather outdoor illumination. The key to our approach is a novel dual attention autoencoder (DAA) with two independent branches to compress the sun and sky lighting information from an input HDR panorama, respectively. This enables more accurate lighting estimation as evidenced by our experiments since the mutual interference between the sun and sky can be precluded effectively. In DAA, we design the adaptive feature pyramid and the attention module to promote its accuracy in compression. We further develop a sun-sky predictor, a masked network, to learn the sun and sky lighting conditions from an FoV-limited image. Comprehensive qualitative and quantitative experiments verify the effectiveness of our proposed approach and show its superiority over the state of the arts.