Photographic Appearance Enhancement via Detail-Based Dictionary Learning

A number of edge-aware filters can efficiently boost the appearance of an image by detail decomposition and enhancement. However, they often fail to produce photographic enhanced appearance due to some visible artifacts, especially noise, halos and unnatural contrast. The essential reason is that the guidance and the constraint of high-quality appearance are not sufficient enough in the process of enhancement. Thus our idea is to train a detail dictionary from a lot of high-quality patches in order to constrain and control the entire appearance enhancement. In this paper, we propose a novel learningbased enhancement method for photographic appearance, which includes two main stages: dictionary training and sparse reconstruction. In the training stage, we construct a training set of detail patches extracted from some high-quality photos, and then train an overcomplete detail dictionary by iteratively minimizing an ℓ1-norm energy function. In the reconstruction stage, we employ the trained dictionary to reconstruct the boosted detail layer, and further formalize a gradient-guided optimization function to improve the local coherence between patches. Moreover, we propose two evaluation metrics to measure the performance of appearance enhancement. The final experimental results have demonstrated the effectiveness of our learning-based enhancement method.

[1]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[2]  Cewu Lu,et al.  Image smoothing via L0 gradient minimization , 2011, ACM Trans. Graph..

[3]  Narendra Ahuja,et al.  Real-time O(1) bilateral filtering , 2009, CVPR.

[4]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[5]  Diego Gutierrez,et al.  Computational Simulation of Alternative Photographic Processes , 2013, Comput. Graph. Forum.

[6]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[7]  Fatih Porikli,et al.  Constant time O(1) bilateral filtering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[9]  Narendra Ahuja,et al.  Real-time O(1) bilateral filtering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Dani Lischinski,et al.  Content‐Aware Automatic Photo Enhancement , 2012, Comput. Graph. Forum.

[11]  Wei Liu,et al.  Dictionary Pair Learning on Grassmann Manifolds for Image Denoising , 2015, IEEE Transactions on Image Processing.

[12]  Dao-Qing Dai,et al.  Structured Sparse Error Coding for Face Recognition With Occlusion , 2013, IEEE Transactions on Image Processing.

[13]  Yunjin Lee,et al.  Art‐photographic detail enhancement , 2014, Comput. Graph. Forum.

[14]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[15]  Ben Weiss,et al.  Fast median and bilateral filtering , 2006, ACM Trans. Graph..

[16]  Rafael C. González,et al.  Digital image processing, 3rd Edition , 2008 .

[17]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[18]  Jiawen Chen,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, SIGGRAPH 2007.

[19]  Donghua Zhou,et al.  Single image haze removal via depth-based contrast stretching transform , 2014, Science China Information Sciences.

[20]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, SIGGRAPH 2006.

[21]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[22]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[23]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, SIGGRAPH 2008.

[24]  Lizhuang Ma,et al.  Edge-preserving image decomposition via joint weighted least squares , 2015, Computational Visual Media.

[25]  Frédo Durand,et al.  Two-scale tone management for photographic look , 2006, SIGGRAPH 2006.

[26]  Turgay Çelik,et al.  Two-dimensional histogram equalization and contrast enhancement , 2012, Pattern Recognit..

[27]  张帆,et al.  Enlarging Image by Constrained Least Square Approach with Shape Preserving , 2015 .

[28]  Renjie Liao,et al.  Deep Edge-Aware Filters , 2015, ICML.

[29]  Yücel Altunbasak,et al.  A Histogram Modification Framework and Its Application for Image Contrast Enhancement , 2009, IEEE Transactions on Image Processing.

[30]  Jianping Hu,et al.  Least-squares images for edge-preserving smoothing , 2015, Computational Visual Media.

[31]  Dani Lischinski,et al.  Personalization of image enhancement , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Michael Elad,et al.  Learning Multiscale Sparse Representations for Image and Video Restoration , 2007, Multiscale Model. Simul..

[33]  Ashish Kapoor,et al.  Collaborative personalization of image enhancement , 2011, CVPR.

[34]  Turgay Çelik,et al.  Contextual and Variational Contrast Enhancement , 2011, IEEE Transactions on Image Processing.

[35]  Maneesh Agrawala,et al.  Multiscale shape and detail enhancement from multi-light image collections , 2007, SIGGRAPH 2007.

[36]  Jan Kautz,et al.  Local Laplacian filters: edge-aware image processing with a Laplacian pyramid , 2011, SIGGRAPH 2011.

[37]  Liu Jingyuan,et al.  A selective overview of feature screening for ultrahigh-dimensional data , 2015, Science China Mathematics.

[38]  Qing Zhang,et al.  Underexposed Video Enhancement via Perception-Driven Progressive Fusion , 2016, IEEE Transactions on Visualization and Computer Graphics.

[39]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[41]  Aljoscha Smolic,et al.  Automated Aesthetic Analysis of Photographic Images , 2015, IEEE Transactions on Visualization and Computer Graphics.

[42]  Frédo Durand,et al.  Edge-preserving multiscale image decomposition based on local extrema , 2009, ACM Trans. Graph..

[43]  J. Alex Stark,et al.  Adaptive image contrast enhancement using generalizations of histogram equalization , 2000, IEEE Trans. Image Process..

[44]  Manuel M. Oliveira,et al.  Domain transform for edge-aware image and video processing , 2011, SIGGRAPH 2011.

[45]  Raanan Fattal,et al.  Edge-avoiding wavelets and their applications , 2009, ACM Trans. Graph..

[46]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[47]  Haifeng Li,et al.  Error aware multiple vertical planes based visual localization for mobile robots in urban environments , 2015, Science China Information Sciences.

[48]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[49]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[50]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[51]  Zeev Farbman,et al.  Interactive local adjustment of tonal values , 2006, ACM Trans. Graph..

[52]  Ralph R. Martin,et al.  Mixed-Domain Edge-Aware Image Manipulation , 2013, IEEE Transactions on Image Processing.

[53]  Paul L. Rosin,et al.  Intelligent Visual Media Processing: When Graphics Meets Vision , 2017, Journal of Computer Science and Technology.

[54]  Qi Zhang,et al.  Rolling Guidance Filter , 2014, ECCV.

[55]  Edward H. Adelson,et al.  Personal photo enhancement using example images , 2010, TOGS.