SRFlow-DA: Super-Resolution Using Normalizing Flow with Deep Convolutional Block

Multiple high-resolution (HR) images can be generated from a single low-resolution (LR) image, as super-resolution (SR) is an underdetermined problem. Recently, the conditional normalizing flow-based model, SRFlow, shows remarkable performance by learning an exact map-ping from HR image manifold to a latent space. The flow-based SR model allows sampling multiple output images from a learned SR space with a given LR image. In this work, we propose SRFlow-DA which has a more suitable architecture for the SR task based on the original SRFlow model. Specifically, our approach enlarges the receptive field by stacking more convolutional layers in the affine couplings, and so our model can get more expressive power. At the same time, we reduce the total number of model pa-rameters for efficiency. Compared to SRFlow, our SRFlow-DA achieves better or comparable PSNR and LPIPS for × 4 and ×8 SR tasks, while having a reduced number of parameters. In addition, our method generates visually clear results without excessive sharpness artifacts.

[1]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yoshua Bengio,et al.  NICE: Non-linear Independent Components Estimation , 2014, ICLR.

[3]  Prafulla Dhariwal,et al.  Glow: Generative Flow with Invertible 1x1 Convolutions , 2018, NeurIPS.

[4]  Seungyong Lee,et al.  SRFeat: Single Image Super-Resolution with Feature Discrimination , 2018, ECCV.

[5]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[6]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[7]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Max Welling,et al.  Learning Likelihoods with Conditional Normalizing Flows , 2019, ArXiv.

[11]  Yu Qiao,et al.  RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  L. Gool,et al.  SRFlow: Learning the Super-Resolution Space with Normalizing Flow , 2020, ECCV.

[14]  Hazim Kemal Ekenel,et al.  SROBB: Targeted Perceptual Loss for Single Image Super-Resolution , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Kun Gao,et al.  NTIRE 2021 Learning the Super-Resolution Space Challenge , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Radu Timofte,et al.  DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution , 2020, ArXiv.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Jie Zhou,et al.  Structure-Preserving Super Resolution With Gradient Guidance , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Michal Irani,et al.  "Zero-Shot" Super-Resolution Using Deep Internal Learning , 2017, CVPR.

[22]  T. Michaeli,et al.  Explorable Super Resolution , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Yochai Blau,et al.  The Perception-Distortion Tradeoff , 2017, CVPR.

[25]  Nam Ik Cho,et al.  Natural and Realistic Single Image Super-Resolution With Explicit Natural Manifold Discrimination , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[27]  Nam Ik Cho,et al.  Meta-Transfer Learning for Zero-Shot Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).