Evaluating Parameterization Methods for Convolutional Neural Network (CNN)-Based Image Operators

Recently, deep neural networks have been widely used to approximate or improve image operators. In general, an image operator has some hyper-parameters that change its operating configurations, e.g., the strength of smoothing, up-scale factors in super-resolution, or a type of image operator. To address varying parameter settings, an image operator taking such parameters as its input, namely a parameterized image operator, is an essential cue in image processing. Since many types of parameterization techniques exist, a comparative analysis is required in the context of image processing. In this paper, we therefore analytically explore the operation principles of these parameterization techniques and study their differences. In addition, performance comparisons between image operators parameterized by using these methods are assessed experimentally on common image processing tasks including image smoothing, denoising, deblocking, and super-resolution.

[1]  Li Xu,et al.  Structure extraction from texture via relative total variation , 2012, ACM Trans. Graph..

[2]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[3]  Jiaolong Yang,et al.  A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing (Supplementary Material) , 2017 .

[4]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[6]  Weisi Lin,et al.  Efficient Image Deblocking Based on Postfiltering in Shifted Windows , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  XuYi,et al.  Image smoothing via L0 gradient minimization , 2011 .

[9]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[10]  Cewu Lu,et al.  Combining sketch and tone for pencil drawing production , 2012, NPAR '12.

[11]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Renjie Liao,et al.  Deep Edge-Aware Filters , 2015, ICML.

[13]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[15]  Nenghai Yu,et al.  Decouple Learning for Parameterized Image Operators , 2018 .

[16]  Lei Zhang,et al.  Waterloo Exploration Database: New Challenges for Image Quality Assessment Models , 2017, IEEE Transactions on Image Processing.

[17]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  Jia Xu,et al.  Fast Image Processing with Fully-Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.