Efficient Data Optimisation for Harmonic Inpainting with Finite Elements

Harmonic inpainting with optimised data is very popular for inpainting-based image compression. We improve this approach in three important aspects. Firstly, we replace the standard finite differences discretisation by a finite element method with triangle elements. This does not only speed up inpainting and data selection, but even improves the reconstruction quality. Secondly, we propose highly efficient algorithms for spatial and tonal data optimisation that are several orders of magnitude faster than state-of-the-art methods. Last but not least, we show that our algorithms also allow working with very large images. This has previously been impractical due to the memory and runtime requirements of prior algorithms.

[1]  Martin Rumpf,et al.  An Adaptive Finite Element Method for Large Scale Image Processing , 1999, J. Vis. Commun. Image Represent..

[2]  Claes Johnson.  Numerical solution of partial differential equations by the finite element method , 1988 .

[3]  J. Kacur,et al.  Solution of nonlinear diffusion appearing in image smoothing and edge detection , 1995 .

[4]  A. R. Mitchell,et al.  The Finite Difference Method in Partial Differential Equations , 1980 .

[5]  Pawan Sinha,et al.  A Perceptually Based Comparison of Image Similarity Metrics , 2011, Perception.

[6]  Frank Neumann,et al.  Optimising Spatial and Tonal Data for PDE-based Inpainting , 2015, ArXiv.

[7]  Dong Liu,et al.  Image Compression With Edge-Based Inpainting , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[9]  Jonathan Richard Shewchuk,et al.  What is a Good Linear Element? Interpolation, Conditioning, and Quality Measures , 2002, IMR.

[10]  I. Loris,et al.  On the convergence of variable metric line-search based proximal-gradient method under the Kurdyka-Lojasiewicz inequality , 2016 .

[11]  Joachim Weickert,et al.  Why Does Non-binary Mask Optimisation Work for Diffusion-Based Image Compression? , 2015, EMMCVPR.

[12]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[13]  Yuri A. Melnikov,et al.  Green's Functions: Construction and Applications , 2012 .

[14]  Franco P. Preparata,et al.  Computational Geometry , 1985, Texts and Monographs in Computer Science.

[15]  Sebastian Hoffmann,et al.  Competitive image compression with linear PDEs , 2017 .

[16]  S. Boujena,et al.  An Improved Nonlinear Model for Image Inpainting , 2015 .

[17]  Joachim Weickert,et al.  An Optimal Control Approach to Find Sparse Data for Laplace Interpolation , 2013, EMMCVPR.

[18]  Yunjin Chen,et al.  A bi-level view of inpainting - based image compression , 2014, ArXiv.

[19]  Advanced Concepts for Intelligent Vision Systems , 2010, Lecture Notes in Computer Science.

[20]  Joachim Weickert,et al.  Optimising Data for Exemplar-Based Inpainting , 2018, ACIVS.

[21]  Shumeet Baluja,et al.  Representing Images in 200 Bytes: Compression via Triangulation , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[22]  Nira Dyn,et al.  Image compression by linear splines over adaptive triangulations , 2006, Signal Process..

[23]  Aggelos K. Katsaggelos,et al.  Adaptive Image Sampling Using Deep Learning and Its Application on X-Ray Fluorescence Image Reconstruction , 2018, IEEE Transactions on Multimedia.

[24]  Joachim Weickert,et al.  Understanding, Optimising, and Extending Data Compression with Anisotropic Diffusion , 2014, International Journal of Computer Vision.

[25]  Frank Neumann,et al.  Optimising Spatial and Tonal Data for Homogeneous Diffusion Inpainting , 2011, SSVM.

[26]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[27]  Federica Porta,et al.  On the convergence of a linesearch based proximal-gradient method for nonconvex optimization , 2016, 1605.03791.

[28]  Joachim Weickert,et al.  Sparse Inpainting with Smoothed Particle Hydrodynamics , 2020, SIAM J. Imaging Sci..

[29]  Joachim Weickert,et al.  Inpainting-Based Video Compression in FullHD , 2021, SSVM.

[30]  Hans-Peter Seidel,et al.  Towards PDE-Based Image Compression , 2005, VLSM.

[31]  Riccardo Distasi,et al.  Image compression by B-tree triangular coding , 1997, IEEE Trans. Commun..

[32]  Maher Moakher,et al.  A multiscale fourth‐order model for the image inpainting and low‐dimensional sets recovery , 2016, Mathematical Methods in the Applied Sciences.

[33]  Alfred M. Bruckstein,et al.  Scale Space and Variational Methods in Computer Vision , 2011, Lecture Notes in Computer Science.

[34]  K. Mikula,et al.  A coarsening finite element strategy in image selective smoothing , 1997 .

[35]  Michael D. Adams.  A highly-effective incremental/decremental Delaunay mesh-generation strategy for image representation , 2013, Signal Process..

[36]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Edge-based Compression of Cartoon-like Images with Homogeneous Diffusion Edge-based Compression of Cartoon-like Images with Homogeneous Diffusion Edge-based Compression of Cartoon-like Images with Homogeneous Diffusion , 2022 .