NTIRE 2022 Challenge on Learning the Super-Resolution Space

This paper reviews the NTIRE 2022 challenge on learning the super-Resolution space. This challenge aims to raise awareness that the super-resolution problem is ill-posed. Since many high-resolution images map to the same low-resolution image, we asked the participants to create methods that sample diverse super-resolution from the space of possible high-resolution images given a low-resolution image. For evaluation, we use the same protocol as introduced in the last year’s super-resolution space challenge of NTIRE 2021. We compare the submissions of the participating teams and relate them to the approaches from last year. This challenge contains two tracks: 4× and 8× scale factor. In total, 3 teams competed in the final testing phase.

[1]  M. S. Brown,et al.  NTIRE 2022 Challenge on Night Photography Rendering , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Xin Tao,et al.  NTIRE 2022 Image Inpainting Challenge: Report , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Amir S. Bernat,et al.  NTIRE 2022 Spectral Demosaicing Challenge and Data Set , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Jian Sun,et al.  NTIRE 2022 Burst Super-Resolution Challenge , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Amir S. Bernat,et al.  NTIRE 2022 Spectral Recovery Challenge and Data Set , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Jimmy S. J. Ren,et al.  NTIRE 2022 Challenge on Perceptual Image Quality Assessment , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  An Gia Vien,et al.  NTIRE 2022 Challenge on High Dynamic Range Imaging: Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Syed Waqas Zamir,et al.  NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Thuong Nguyen Canh,et al.  NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  R. Timofte,et al.  NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  L. Gool,et al.  RePaint: Inpainting using Denoising Diffusion Probabilistic Models , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Prafulla Dhariwal,et al.  GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models , 2021, ICML.

[13]  David J. Fleet,et al.  Palette: Image-to-Image Diffusion Models , 2021, SIGGRAPH.

[14]  Luc Van Gool,et al.  Normalizing Flow as a Flexible Fidelity Objective for Photo-Realistic Super-resolution , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[15]  S. Ermon,et al.  SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations , 2021, ICLR.

[16]  Eliya Nachmani,et al.  Denoising Diffusion Gamma Models , 2021, ArXiv.

[17]  Luc Van Gool,et al.  Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Youngjune Gwon,et al.  ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Wan-Chi Siu,et al.  Variational AutoEncoder for Reference based Image Super-Resolution , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Kun Gao,et al.  NTIRE 2021 Learning the Super-Resolution Space Challenge , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Younggeun Kim,et al.  Noise Conditional Flow Model for Learning the Super-Resolution Space , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Younghyun Jo,et al.  SRFlow-DA: Super-Resolution Using Normalizing Flow with Deep Convolutional Block , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Prafulla Dhariwal,et al.  Diffusion Models Beat GANs on Image Synthesis , 2021, NeurIPS.

[24]  Xiahai Zhuang,et al.  VSpSR: Explorable Super-Resolution via Variational Sparse Representation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Prafulla Dhariwal,et al.  Improved Denoising Diffusion Probabilistic Models , 2021, ICML.

[26]  Luc Van Gool,et al.  DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  L. Gool,et al.  SRFlow: Learning the Super-Resolution Space with Normalizing Flow , 2020, ECCV.

[28]  Pieter Abbeel,et al.  Denoising Diffusion Probabilistic Models , 2020, NeurIPS.

[29]  Joun Yeop Lee,et al.  SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds , 2020, NeurIPS.

[30]  Marcel C. Bühler,et al.  DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution , 2020, ACCV.

[31]  Tianhao Zhang,et al.  Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimation , 2020, International Journal of Computer Vision.

[32]  T. Michaeli,et al.  Explorable Super Resolution , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Yui-Lam Chan,et al.  AIM 2019 Challenge on Image Extreme Super-Resolution: Methods and Results , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[34]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[35]  Prafulla Dhariwal,et al.  Glow: Generative Flow with Invertible 1x1 Convolutions , 2018, NeurIPS.

[36]  Kyung-Ah Sohn,et al.  Image Super-Resolution via Progressive Cascading Residual Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[37]  Ming Qin,et al.  Densely Connected High Order Residual Network for Single Frame Image Super Resolution , 2018, ArXiv.

[38]  Kyung-Ah Sohn,et al.  Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network , 2018, ECCV.

[39]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[42]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[43]  Thomas S. Huang,et al.  Balanced Two-Stage Residual Networks for Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[44]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Shakir Mohamed,et al.  Learning in Implicit Generative Models , 2016, ArXiv.

[46]  Xin Yu,et al.  Ultra-Resolving Face Images by Discriminative Generative Networks , 2016, ECCV.

[47]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Luc Van Gool,et al.  Seven Ways to Improve Example-Based Single Image Super Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Luc Van Gool,et al.  Jointly Optimized Regressors for Image Super‐resolution , 2015, Comput. Graph. Forum.

[52]  Surya Ganguli,et al.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics , 2015, ICML.

[53]  Yoshua Bengio,et al.  NICE: Non-linear Independent Components Estimation , 2014, ICLR.

[54]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[55]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[56]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[57]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[58]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[60]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[61]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..