Wasserstein Barycenter and Its Application to Texture Mixing

This paper proposes a new definition of the averaging of discrete probability distributions as a barycenter over the Monge-Kantorovich optimal transport space. To overcome the time complexity involved by the numerical solving of such problem, the original Wasserstein metric is replaced by a sliced approximation over 1D distributions. This enables us to introduce a new fast gradient descent algorithm to compute Wasserstein barycenters of point clouds. This new notion of barycenter of probabilities is likely to find applications in computer vision where one wants to average features defined as distributions. We show an application to texture synthesis and mixing, where a texture is characterized by the distribution of the response to a multi-scale oriented filter bank. This leads to a simple way to navigate over a convex domain of color textures.

[1]  D. Dowson,et al.  The Fréchet distance between multivariate normal distributions , 1982 .

[2]  Ken Perlin,et al.  [Computer Graphics]: Three-Dimensional Graphics and Realism , 2022 .

[3]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[4]  J. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, SIGGRAPH.

[5]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[6]  W. Gangbo,et al.  Optimal maps for the multidimensional Monge-Kantorovich problem , 1998 .

[7]  Rupert Paget,et al.  Texture synthesis via a noncausal nonparametric multiscale Markov random field , 1998, IEEE Trans. Image Process..

[8]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[9]  Gershon Elber,et al.  Image Morphing with Feature Preserving Texture , 1999, Comput. Graph. Forum.

[10]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[11]  D. Mumford,et al.  Stochastic models for generic images , 2001 .

[12]  Dani Lischinski,et al.  Texture Mixing and Texture Movie Synthesis Using Statistical Learning , 2001, IEEE Trans. Vis. Comput. Graph..

[13]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[14]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[15]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[16]  Ce Liu,et al.  Figure 1. A texture metamorphosis sequence is generated from the leftmost to the rightmost. Pattern-based Texture Metamorphosis , 2002 .

[17]  Harry Shum,et al.  Pattern-based texture metamorphosis , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[18]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[19]  C. Villani.  Topics in Optimal Transportation , 2003 .

[20]  C. Villani,et al.  Optimal Transportation and Applications , 2003 .

[21]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[22]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[23]  Marcelo Walter,et al.  Texture metamorphosis driven by texton masks , 2005, Comput. Graph..

[24]  Sylvain Lefebvre,et al.  Parallel controllable texture synthesis , 2005, ACM Trans. Graph..

[25]  Robert L Cook,et al.  Wavelet noise , 2005, ACM Trans. Graph..

[26]  Irfan Essa,et al.  Texture optimization for example-based synthesis , 2005, SIGGRAPH 2005.

[27]  F. Durand,et al.  Texture design using a simplicial complex of morphable textures , 2005, SIGGRAPH 2005.

[28]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[29]  Julie Delon,et al.  Movie and video scale-time equalization application to flicker reduction , 2006, IEEE Transactions on Image Processing.

[30]  Anil Kokaram,et al.  The linear Monge-Kantorovitch linear colour mapping for example-based colour transfer , 2007 .

[31]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..

[32]  Mauro Dell'Amico,et al.  8. Quadratic Assignment Problems: Algorithms , 2009 .

[33]  Julien Rabin,et al.  Geodesic Shape Retrieval via Optimal Mass Transport , 2010, ECCV.

[34]  Gabriel Peyré,et al.  Texture Synthesis with Grouplets , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Thomas Deselaers,et al.  ClassCut for Unsupervised Class Segmentation , 2010, ECCV.

[36]  Allen R. Tannenbaum,et al.  Texture Mapping via Optimal Mass Transport , 2010, IEEE Transactions on Visualization and Computer Graphics.

[37]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..