NTIRE 2022 Challenge on High Dynamic Range Imaging: Methods and Results

This paper reviews the challenge on constrained high dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2022. This manuscript focuses on the competition set-up, datasets, the proposed methods and their results. The challenge aims at estimating an HDR image from multiple respective low dynamic range (LDR) observations, which might suffer from under-or over-exposed regions and different sources of noise. The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i.e. solutions can not exceed a given number of operations). In Track 2, participants are asked to minimize the complexity of their solutions while imposing a constraint on fidelity scores (i.e. solutions are required to obtain a higher fidelity score than the prescribed baseline). Both tracks use the same data and metrics: Fidelity is measured by means of PSNR with respect to a ground-truth HDR image (computed both directly and with a canonical tonemapping operation), while complexity metrics include the number of Multiply-Accumulate (MAC) operations and runtime (in seconds).

[1]  M. S. Brown,et al.  NTIRE 2022 Challenge on Night Photography Rendering , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Xin Tao,et al.  NTIRE 2022 Image Inpainting Challenge: Report , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  R. Timofte,et al.  NTIRE 2022 Challenge on Learning the Super-Resolution Space , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  An Gia Vien,et al.  Bidirectional Motion Estimation with Cyclic Cost Volume for High Dynamic Range Imaging , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Yi Cao,et al.  Gamma-enhanced Spatial Attention Network for Efficient High Dynamic Range Imaging , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Yanning Zhang,et al.  A Lightweight Network for High Dynamic Range Imaging , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Amir S. Bernat,et al.  NTIRE 2022 Spectral Demosaicing Challenge and Data Set , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Jian Sun,et al.  NTIRE 2022 Burst Super-Resolution Challenge , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Amir S. Bernat,et al.  NTIRE 2022 Spectral Recovery Challenge and Data Set , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Jimmy S. J. Ren,et al.  NTIRE 2022 Challenge on Perceptual Image Quality Assessment , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Peter Schneider-Kamp,et al.  DRHDR: A Dual branch Residual Network for Multi-Bracket High Dynamic Range Imaging , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Syed Waqas Zamir,et al.  NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  R. Timofte,et al.  NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Thuong Nguyen Canh,et al.  NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Gaocheng Yu,et al.  Efficient Progressive High Dynamic Range Image Restoration via Attention and Alignment Network , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Luc Van Gool,et al.  Revisiting RCAN: Improved Training for Image Super-Resolution , 2022, ArXiv.

[17]  Trevor Darrell,et al.  A ConvNet for the 2020s , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Syed Waqas Zamir,et al.  Restormer: Efficient Transformer for High-Resolution Image Restoration , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Vikas Chandra,et al.  Feature-Align Network and Knowledge Distillation for Efficient Denoising , 2021, ArXiv.

[20]  Luc Van Gool,et al.  SwinIR: Image Restoration Using Swin Transformer , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[21]  Sung-Jea Ko,et al.  Rethinking Coarse-to-Fine Approach in Single Image Deblurring , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Radu Timofte,et al.  NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Jenq-Neng Hwang,et al.  NTIRE 2021 Multi-modal Aerial View Object Classification Challenge , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Yu Qiao,et al.  HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Haoqiang Fan,et al.  ADNet: Attention-guided Deformable Convolutional Network for High Dynamic Range Imaging , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Yibin Huang,et al.  Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Jiashi Feng,et al.  Coordinate Attention for Efficient Mobile Network Design , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[29]  Jie Liu,et al.  Residual Feature Distillation Network for Lightweight Image Super-Resolution , 2020, ECCV Workshops.

[30]  Yongwoo Kim,et al.  Multi-attention Based Ultra Lightweight Image Super-Resolution , 2020, ECCV Workshops.

[31]  Jie Tang,et al.  Residual Feature Aggregation Network for Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Chang Xu,et al.  GhostNet: More Features From Cheap Operations , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  R. Venkatesh Babu,et al.  Towards Practical and Efficient High-Resolution HDR Deghosting with CNN , 2020, ECCV.

[34]  Chen Change Loy,et al.  EDVR: Video Restoration With Enhanced Deformable Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Yanning Zhang,et al.  Attention-Guided Network for Ghost-Free High Dynamic Range Imaging , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Sung-Fang Tsai,et al.  Real- Time Video Denoising on Mobile Phones , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[38]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[39]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Chi-Keung Tang,et al.  End-to-End Deep HDR Imaging with Large Foreground Motions , 2017, ArXiv.

[42]  Ravi Ramamoorthi,et al.  Deep high dynamic range imaging of dynamic scenes , 2017, ACM Trans. Graph..

[43]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[44]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[47]  Harald Brendel,et al.  Creating cinematic wide gamut HDR-video for the evaluation of tone mapping operators and HDR-displays , 2014, Electronic Imaging.

[48]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[49]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[50]  Frédo Durand,et al.  Noise-optimal capture for high dynamic range photography , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[52]  Jan Kautz,et al.  Exposure Fusion , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[53]  Shree K. Nayar,et al.  Optical Splitting Trees for High-Precision Monocular Imaging , 2007, IEEE Computer Graphics and Applications.