Image Compression With Edge-Based Inpainting

In this paper, image compression utilizing visual redundancy is investigated. Inspired by recent advancements in image inpainting techniques, we propose an image compression framework towards visual quality rather than pixel-wise fidelity. In this framework, an original image is analyzed at the encoder side so that portions of the image are intentionally and automatically skipped. Instead, some information is extracted from these skipped regions and delivered to the decoder as assistant information in the compressed fashion. The delivered assistant information plays a key role in the proposed framework because it guides image inpainting to accurately restore these regions at the decoder side. Moreover, to fully take advantage of the assistant information, a compression-oriented edge-based inpainting algorithm is proposed for image restoration, integrating pixel-wise structure propagation and patch-wise texture synthesis. We also construct a practical system to verify the effectiveness of the compression approach in which edge map serves as assistant information and the edge extraction and region removal approaches are developed accordingly. Evaluations have been made in comparison with baseline JPEG and standard MPEG-4 AVC/H.264 intra-picture coding. Experimental results show that our system achieves up to 44% and 33% bits-savings, respectively, at similar visual quality levels. Our proposed framework is a promising exploration towards future image and video compression.

[1]  Anil K. Jain,et al.  Texture Analysis , 2018, Handbook of Image Processing and Computer Vision.

[2]  Robert J. Safranek,et al.  Signal compression based on models of human perception , 1993, Proc. IEEE.

[3]  J. Mundy,et al.  Driving vision by topology , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[4]  Norman D. Black,et al.  Second-generation image coding: an overview , 1997, CSUR.

[5]  Luigi Atzori,et al.  Error concealment in video transmission over packet networks by a sketch-based approach , 1999, Signal Process. Image Commun..

[6]  Wenjun Zeng,et al.  Geometric-structure-based error concealment with novel applications in block-based low-bit-rate coding , 1999, IEEE Trans. Circuits Syst. Video Technol..

[7]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[8]  Lina J. Karam,et al.  Locally adaptive perceptual image coding , 2000, IEEE Trans. Image Process..

[9]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[10]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[11]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[12]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[13]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[14]  Baining Guo,et al.  Real-time texture synthesis by patch-based sampling , 2001, TOGS.

[15]  Tony F. Chan,et al.  Non-texture inpainting by curvature-driven diffusions (CDD) , 2001 .

[16]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[17]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[18]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[19]  Brendan J. Frey,et al.  Epitomic analysis of appearance and shape , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Song-Chun Zhu,et al.  Towards a mathematical theory of primal sketch and sketchability , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[22]  Daniel Cohen-Or,et al.  Fragment-based image completion , 2003, ACM Trans. Graph..

[23]  Guillermo Sapiro,et al.  Structure and texture filling-in of missing image blocks in wireless transmission and compression applications , 2003, IEEE Trans. Image Process..

[24]  Chi-Keung Tang,et al.  Image repairing: robust image synthesis by adaptive ND tensor voting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[26]  Harald Grossauer,et al.  A Combined PDE and Texture Synthesis Approach to Inpainting , 2004, ECCV.

[27]  Andrew Blake,et al.  PatchWorks: Example-Based Region Tiling for Image Editing , 2004 .

[28]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[29]  Sylvain Lefebvre,et al.  Parallel controllable texture synthesis , 2005, ACM Trans. Graph..

[30]  Brendan J. Frey,et al.  Video Epitomes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Nipun Kwatra,et al.  Texture optimization for example-based synthesis , 2005, ACM Trans. Graph..

[32]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[33]  Rachid Deriche,et al.  Vector-valued image regularization with PDEs: a common framework for different applications , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Chen Wang,et al.  Image compression with structure-aware inpainting , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[35]  Feng Wu,et al.  Compression with vision technologies , 2006 .

[36]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..