Zerotree Wavelet Based Image Quilting for Fast Texture Synthesis

In this paper we propose a fast DWT based multi-resolution texture synthesis algorithm in which coefficient blocks of the spatio-frequeny components of the input texture are efficiently stitched together (quilted) to form the corresponding components of the synthesised output texture. We propose the use of an automatically generated threshold to determine the significant coefficients which acts as elements of a matching template used in the texture quilting process. We show that the use of a limited set of, visually significant coefficients, regardless of their level of resolution, not only reduces the computational cost, but also results in more realistic texture synthesis. We use popular test textures to compare our results with that of the existing state-or-the-art techniques. Many application scenarios of the proposed algorithm are also discussed.

[1]  W. Sweldens The Lifting Scheme: A Custom - Design Construction of Biorthogonal Wavelets "Industrial Mathematics , 1996 .

[2]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[4]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[5]  I. Daubechies,et al.  Tree Approximation and Optimal Encoding , 2001 .

[6]  Siwei Lyu,et al.  A digital technique for art authentication , 2004, Proc. Natl. Acad. Sci. USA.

[7]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[8]  S. Mallat,et al.  Thresholding estimators for linear inverse problems and deconvolutions , 2003 .

[9]  Gabor T. Herman,et al.  Image reconstruction from projections : the fundamentals of computerized tomography , 1980 .

[10]  Hans-Peter Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, ACM Trans. Graph..

[11]  Rupert Paget,et al.  Texture synthesis via a noncausal nonparametric multiscale Markov random field , 1998, IEEE Trans. Image Process..

[12]  Peter Schröder,et al.  Spherical wavelets: efficiently representing functions on the sphere , 1995, SIGGRAPH.

[13]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[14]  Stefan Carlsson,et al.  Sketch based coding of grey level images , 1988 .

[15]  A. N. Tikhonov,et al.  The regularization of ill-posed problems , 1963 .

[16]  E. Candès,et al.  The curvelet representation of wave propagators is optimally sparse , 2004, math/0407210.

[17]  S. Laughlin A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[18]  J. Cowan,et al.  The functional geometry of local and horizontal connections in a model of V1 , 2003, Journal of Physiology-Paris.

[19]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Eero P. Simoncelli,et al.  Image compression via joint statistical characterization in the wavelet domain , 1999, IEEE Trans. Image Process..

[21]  Eran A. Edirisinghe,et al.  Fast wavelet transform domain texture synthesis , 2004, IS&T/SPIE Electronic Imaging.

[22]  Andrei Khodakovsky,et al.  Compression of Normal Meshes , 2004 .

[23]  R. Millane,et al.  Effects of occlusion, edges, and scaling on the power spectra of natural images. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  D. Donoho,et al.  Translation-Invariant De-Noising , 1995 .

[25]  Hugues Hoppe,et al.  Shape Compression using Spherical Geometry Images , 2005, Advances in Multiresolution for Geometric Modelling.

[26]  David L. Donoho,et al.  Curvelets and reconstruction of images from noisy radon data , 2000, SPIE Optics + Photonics.

[27]  Hujun Bao,et al.  Visual simulation of weathering by γ-ton tracing , 2005, ACM Trans. Graph..

[28]  Justin K. Romberg,et al.  Wavelet-domain approximation and compression of piecewise smooth images , 2006, IEEE Transactions on Image Processing.

[29]  Ken Perlin,et al.  An image synthesizer , 1988 .

[30]  O. Houser,et al.  Radiology of the Skull and Brain , 1976 .

[31]  Song-Chun Zhu,et al.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998, International Journal of Computer Vision.

[32]  Jean-Michel Morel,et al.  Denoising image sequences does not require motion estimation , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[33]  Richard G. Baraniuk,et al.  The multiscale structure of non-differentiable image manifolds , 2005, SPIE Optics + Photonics.

[34]  F. O’Sullivan A Statistical Perspective on Ill-posed Inverse Problems , 1986 .

[35]  Jelena Kovacevic,et al.  Wavelets and Subband Coding , 2013, Prentice Hall Signal Processing Series.

[36]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[37]  A. Tsybakov,et al.  Minimax theory of image reconstruction , 1993 .

[38]  Aaron Hertzmann,et al.  Illustrating smooth surfaces , 2000, SIGGRAPH.

[39]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[40]  Stéphane Mallat,et al.  Bandelet Image Approximation and Compression , 2005, Multiscale Model. Simul..

[41]  Edward H. Adelson,et al.  Compressing and companding high dynamic range images with subband architectures , 2005, ACM Trans. Graph..

[42]  Paolo Cignoni,et al.  Metro: Measuring Error on Simplified Surfaces , 1998, Comput. Graph. Forum.

[43]  E. Candès,et al.  Astronomical image representation by the curvelet transform , 2003, Astronomy & Astrophysics.

[44]  Donald S. Fussell,et al.  Computer rendering of stochastic models , 1982, Commun. ACM.

[45]  Harry Shum,et al.  Pattern-based texture metamorphosis , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[46]  Ronald R. Coifman,et al.  Brushlets: A Tool for Directional Image Analysis and Image Compression , 1997 .

[47]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[48]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[49]  E. Candès,et al.  Recovering edges in ill-posed inverse problems: optimality of curvelet frames , 2002 .

[50]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[51]  Gauthier Lafruit,et al.  MESHGRID-a compact, multiscalable and animation-friendly surface representation , 2004, IEEE Trans. Circuits Syst. Video Technol..

[52]  S. Mallat,et al.  Surface compression with geometric bandelets , 2005, ACM Trans. Graph..

[53]  Song-Chun Zhu,et al.  Prior Learning and Gibbs Reaction-Diffusion , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Michael T. Orchard,et al.  Image coding based on a morphological representation of wavelet data , 1999, IEEE Trans. Image Process..

[55]  Béatrice Pesquet-Popescu,et al.  Three-dimensional lifting schemes for motion compensated video compression , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[56]  Mark S. Peercy,et al.  Efficient bump mapping hardware , 1997, SIGGRAPH.

[57]  Denis Zorin,et al.  Digital geometry processing , 2001 .

[58]  J. Morel,et al.  On image denoising methods , 2004 .

[59]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[60]  Yanxi Liu,et al.  Near-regular texture analysis and manipulation , 2004, SIGGRAPH 2004.

[61]  Rangasami L. Kashyap,et al.  Synthesis and Estimation of Random Fields Using Long-Correlation Models , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Pierre Moulin,et al.  Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients , 2001, IEEE Trans. Image Process..

[63]  Christine Guillemot,et al.  Oriented Wavelet Transform for Image Compression and Denoising , 2006, IEEE Transactions on Image Processing.

[64]  Y. Meyer Principe d'incertitude, bases hilbertiennes et algèbres d'opérateurs , 1986 .

[65]  Tony F. Chan,et al.  Variational Restoration of Nonflat Image Features: Models and Algorithms , 2001, SIAM J. Appl. Math..

[66]  Jarke J. van Wijk,et al.  Image based flow visualization , 2002, ACM Trans. Graph..

[67]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[68]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  J. E. Pearson Complex Patterns in a Simple System , 1993, Science.

[70]  Justin K. Romberg,et al.  Bayesian tree-structured image modeling using wavelet-domain hidden Markov models , 2001, IEEE Trans. Image Process..

[71]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[72]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Baltasar Beferull-Lozano,et al.  Directionlets: anisotropic multidirectional representation with separable filtering , 2006, IEEE Transactions on Image Processing.

[74]  S. Mallat A wavelet tour of signal processing , 1998 .

[75]  Michael T. Orchard,et al.  Space-frequency quantization for wavelet image coding , 1997, IEEE Trans. Image Process..

[76]  Ken Perlin,et al.  Improving noise , 2002, SIGGRAPH.

[77]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[78]  Steven Worley,et al.  A cellular texture basis function , 1996, SIGGRAPH.

[79]  Pierre Alliez,et al.  Variational shape approximation , 2004, ACM Trans. Graph..

[80]  A. Kolmogorov Dissipation of energy in the locally isotropic turbulence , 1941, Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences.

[81]  Kris Popat,et al.  Novel cluster-based probability model for texture synthesis, classification, and compression , 1993, Other Conferences.

[82]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[83]  Rama Chellappa,et al.  Texture synthesis using 2-D noncausal autoregressive models , 1985, IEEE Trans. Acoust. Speech Signal Process..

[84]  D. Mumford,et al.  Stochastic models for generic images , 2001 .

[85]  Justin K. Romberg,et al.  Hidden Markov Tree Models for Complex Wavelet Transforms , 2002 .

[86]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[87]  Thomas Malzbender,et al.  A Survey of Methods for Volumetric Scene Reconstruction from Photographs , 2001, VG.

[88]  M. Vetterli Multi-dimensional sub-band coding: Some theory and algorithms , 1984 .

[89]  Tony DeRose,et al.  Piecewise smooth surface reconstruction , 1994, SIGGRAPH.

[90]  Xin Li On exploiting geometric constraint of image wavelet coefficients , 2003, IEEE Trans. Image Process..

[91]  Frédo Durand,et al.  Texture design using a simplicial complex of morphable textures , 2005, SIGGRAPH '05.

[92]  I. Daubechies,et al.  Biorthogonal bases of compactly supported wavelets , 1992 .

[93]  F. Wolf,et al.  Universality in visual cortical pattern formation , 2003, Journal of Physiology-Paris.

[94]  Tai Sing Lee,et al.  Computations in the early visual cortex , 2003, Journal of Physiology-Paris.

[95]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[96]  Béla Julesz,et al.  Visual Pattern Discrimination , 1962, IRE Trans. Inf. Theory.

[97]  Pietro Perona Orientation diffusions , 1998, IEEE Trans. Image Process..

[98]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[99]  Frank Nielsen,et al.  Volumetric illustration: designing 3D models with internal textures , 2004, ACM Trans. Graph..

[100]  Tony DeRose,et al.  Wavelet noise , 2005, SIGGRAPH 2005.

[101]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[102]  Rachid Deriche,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[103]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[104]  Yanxi Liu,et al.  The Promise and Perils of Near-Regular Texture , 2004, International Journal of Computer Vision.

[105]  Mi-Suen Lee,et al.  A Computational Framework for Segmentation and Grouping , 2000 .

[106]  Richard G. Baraniuk,et al.  High-resolution navigation on non-differentiable image manifolds , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[107]  Andrew Witkin,et al.  Reaction-diffusion textures , 1991, SIGGRAPH.

[108]  David S. Taubman,et al.  Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable video compression , 2003, IEEE Trans. Image Process..

[109]  Baining Guo,et al.  Real-time texture synthesis by patch-based sampling , 2001, TOGS.

[110]  David Mumford,et al.  Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves Model , 2004, International Journal of Computer Vision.

[111]  Peter Lindstrom,et al.  Fast and memory efficient polygonal simplification , 1998, Proceedings Visualization '98 (Cat. No.98CB36276).

[112]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[113]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[114]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[115]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[116]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[117]  Demin Wang,et al.  Curved wavelet transform for image coding , 2006, IEEE Transactions on Image Processing.

[118]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[119]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[120]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[121]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[122]  Yizhou Yu,et al.  Feature matching and deformation for texture synthesis , 2004, ACM Trans. Graph..

[123]  Minh N. Do,et al.  Rat e-distortion optimized tree structured compression algorithms for piecewise smooth images , 2005 .

[124]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[125]  Stéphane Mallat,et al.  Characterization of Signals from Multiscale Edges , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[126]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[127]  David S. Taubman,et al.  High performance scalable image compression with EBCOT , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[128]  N. Kingsbury Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[129]  Anton Alstes Wang Tiles for Image and Texture Generation , 2004 .

[130]  G. J. Burton,et al.  Color and spatial structure in natural scenes. , 1987, Applied optics.

[131]  John P. Lewis,et al.  Texture synthesis for digital painting , 1984, SIGGRAPH.

[132]  E. Candès,et al.  Curvelets and Fourier Integral Operators , 2003 .

[133]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[134]  J. Canny A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[135]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[136]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[137]  Jozsef Vass,et al.  Significance-linked connected component analysis for wavelet image coding , 1999, IEEE Trans. Image Process..

[138]  Jerry L. Prince,et al.  Snakes, shapes, and gradient vector flow , 1998, IEEE Trans. Image Process..

[139]  Paul S. Heckbert,et al.  Survey of Texture Mapping , 1986, IEEE Computer Graphics and Applications.

[140]  Greg Turk,et al.  Generating textures on arbitrary surfaces using reaction-diffusion , 1991, SIGGRAPH.

[141]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[142]  D Kersten,et al.  Predictability and redundancy of natural images. , 1987, Journal of the Optical Society of America. A, Optics and image science.