Structure Adaptive Total Variation Minimization-Based Image Decomposition

Structure-preserving image decomposition separates a given image into structure and texture by smoothing the image, simultaneously preserving or enhancing image edges. The well-studied problem of image decomposition is applied to various areas, such as image smoothing, detail enhancement, non-photorealistic rendering, image artistic rendering, and high-dynamic-range compression. In this paper, we propose a fast algorithm for structure-preserving image decomposition that adopts total variation (TV) minimization to the moving least squares (MLS) method with non-local weights, called structure adaptive TV (SATV) minimization. MLS with non-local weights provides high accuracy approximation that is robust to noise, and allows a fast convergence with TV regularization term. As a result, our proposed SATV preserves the dominant structure while flattening fine-scale details. The experimental results show that the SATV minimization algorithm provides faster and more robust image decomposition than the well-known previous approaches. We demonstrate the usefulness of our algorithm by presenting successful applications in image smoothing and detail enhancement.

[1]  Frédo Durand,et al.  Edge-preserving multiscale image decomposition based on local extrema , 2009, ACM Trans. Graph..

[2]  Philip J. Willis,et al.  Two-level joint local laplacian texture filtering , 2015, The Visual Computer.

[3]  Dani Lischinski,et al.  Digital reconstruction of halftoned color comics , 2012, ACM Trans. Graph..

[4]  Stéphane Mallat,et al.  Sparse geometric image representations with bandelets , 2005, IEEE Transactions on Image Processing.

[5]  Abdeldjalil Ouahabi,et al.  Multifractal analysis for texture characterization: A new approach based on DWT , 2010, 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010).

[6]  Abdeldjalil Ouahabi,et al.  A review of wavelet denoising in medical imaging , 2013, 2013 8th International Workshop on Systems, Signal Processing and their Applications (WoSSPA).

[7]  Qingxiong Yang,et al.  Recursive Bilateral Filtering , 2012, ECCV.

[8]  Jean-François Aujol,et al.  Adaptive Regularization of the NL-Means: Application to Image and Video Denoising , 2014, IEEE Transactions on Image Processing.

[9]  Seungyong Lee,et al.  Flow-Based Image Abstraction , 2009, IEEE Transactions on Visualization and Computer Graphics.

[10]  Frédo Durand,et al.  Two-scale tone management for photographic look , 2006, SIGGRAPH 2006.

[11]  Y. Rao,et al.  Generalized Equalization Model for Image Enhancement , 2016 .

[12]  Manuel M. Oliveira,et al.  Domain transform for edge-aware image and video processing , 2011, SIGGRAPH 2011.

[13]  M. Kass,et al.  Smoothed local histogram filters , 2010, SIGGRAPH 2010.

[14]  Jan Kautz,et al.  Local Laplacian filters , 2015, Commun. ACM.

[15]  Raanan Fattal,et al.  Diffusion maps for edge-aware image editing , 2010, SIGGRAPH 2010.

[16]  Ralph R. Martin,et al.  Online Video Stream Abstraction and Stylization , 2011, IEEE Transactions on Multimedia.

[17]  Raanan Fattal,et al.  Edge-avoiding wavelets and their applications , 2009, ACM Trans. Graph..

[18]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[19]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[20]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[21]  Thomas Pock,et al.  Non-local Total Generalized Variation for Optical Flow Estimation , 2014, ECCV.

[22]  Li-Wei Kang,et al.  Self-Learning Based Image Decomposition With Applications to Single Image Denoising , 2014, IEEE Transactions on Multimedia.

[23]  Seungyong Lee,et al.  Bilateral texture filtering , 2014, ACM Trans. Graph..

[24]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[25]  Yizhou Yu,et al.  Title An L 1 image transform for edge-preserving smoothing andscene-level intrinsic decomposition , 2015 .

[26]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[27]  Li Xu,et al.  Structure extraction from texture via relative total variation , 2012, ACM Trans. Graph..

[28]  Zhen Ji,et al.  Edge-Preserving Texture Suppression Filter Based on Joint Filtering Schemes , 2013, IEEE Transactions on Multimedia.

[29]  Stéphane Mallat,et al.  Bandelet Image Approximation and Compression , 2005, Multiscale Model. Simul..

[30]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Rabab K. Ward,et al.  Visually Favorable Tone-Mapping With High Compression Performance in Bit-Depth Scalable Video Coding , 2013, IEEE Trans. Multim..

[32]  Shiming Xiang,et al.  Segment Graph Based Image Filtering: Fast Structure-Preserving Smoothing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[34]  Hua Huang,et al.  Guided Adaptive Image Smoothing via Directional Anisotropic Structure Measurement , 2015, IEEE Transactions on Visualization and Computer Graphics.

[35]  Qi Zhang,et al.  Rolling Guidance Filter , 2014, ECCV.

[36]  Meng Wang,et al.  PicWords: Render a Picture by Packing Keywords , 2014, IEEE Transactions on Multimedia.

[37]  D. Donoho Wedgelets: nearly minimax estimation of edges , 1999 .

[38]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Jean Ponce,et al.  Robust image filtering using joint static and dynamic guidance , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[41]  Sukho Lee,et al.  A Framework for Moving Least Squares Method with Total Variation Minimizing Regularization , 2013, Journal of Mathematical Imaging and Vision.

[42]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, SIGGRAPH 2008.

[43]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[44]  Jongmin Baek,et al.  Accelerating spatially varying Gaussian filters , 2010, SIGGRAPH 2010.

[45]  Hayder Radha,et al.  Translation-Invariant Contourlet Transform and Its Application to Image Denoising , 2006, IEEE Transactions on Image Processing.

[46]  Maneesh Agrawala,et al.  Multiscale shape and detail enhancement from multi-light image collections , 2007, SIGGRAPH 2007.

[47]  Shi-Min Hu,et al.  Efficient affinity-based edit propagation using K-D tree , 2009, ACM Trans. Graph..

[48]  陈世峰 Style Transfer via Image Component Analysis , 2013 .

[49]  Aykut Erdem,et al.  Structure-preserving image smoothing via region covariances , 2013, ACM Trans. Graph..

[50]  Ching-Te Chiu,et al.  Pseudo-Multiple-Exposure-Based Tone Fusion With Local Region Adjustment , 2015, IEEE Transactions on Multimedia.

[51]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[52]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[53]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[54]  Jean-Michel Morel,et al.  Fast Cartoon + Texture Image Filters , 2010, IEEE Transactions on Image Processing.

[55]  Jean-Michel Morel,et al.  The staircasing effect in neighborhood filters and its solution , 2006, IEEE Transactions on Image Processing.

[56]  Jiawen Chen,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, SIGGRAPH 2007.

[57]  Yehoshua Y. Zeevi,et al.  Variational denoising of partly textured images by spatially varying constraints , 2006, IEEE Transactions on Image Processing.

[58]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[59]  Mila Nikolova,et al.  Weakly Constrained Minimization: Application to the Estimation of Images and Signals Involving Constant Regions , 2004, Journal of Mathematical Imaging and Vision.

[60]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, SIGGRAPH 2006.

[62]  Kemal Ugur,et al.  Efficient MRF Energy Propagation for Video Segmentation via Bilateral Filters , 2013, IEEE Transactions on Multimedia.

[63]  E. Candès,et al.  Recovering edges in ill-posed inverse problems: optimality of curvelet frames , 2002 .

[64]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[65]  Xin Yu,et al.  Efficient Patch-Wise Non-Uniform Deblurring for a Single Image , 2014, IEEE Transactions on Multimedia.

[66]  Cewu Lu,et al.  Image smoothing via L0 gradient minimization , 2011, ACM Trans. Graph..

[67]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[68]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .