Learning Detail Transfer based on Geometric Features

The visual richness of computer graphics applications is frequently limited by the difficulty of obtaining high‐quality, detailed 3D models. This paper proposes a method for realistically transferring details (specifically, displacement maps) from existing high‐quality 3D models to simple shapes that may be created with easy‐to‐learn modeling tools. Our key insight is to use metric learning to find a combination of geometric features that successfully predicts detail‐map similarities on the source mesh; we use the learned feature combination to drive the detail transfer. The latter uses a variant of multi‐resolution non‐parametric texture synthesis, augmented by a high‐frequency detail transfer step in texture space. We demonstrate that our technique can successfully transfer details among a variety of shapes including furniture and clothing.

[1]  James M. Rehg,et al.  Terrain Synthesis from Digital Elevation Models , 2007, IEEE Transactions on Visualization and Computer Graphics.

[2]  Alla Sheffer,et al.  Animation wrinkling: augmenting coarse cloth simulations with realistic-looking wrinkles , 2010, ACM Trans. Graph..

[3]  Daniel Cohen-Or,et al.  Fit and diverse , 2012, ACM Trans. Graph..

[4]  Thomas A. Funkhouser,et al.  Schelling points on 3D surface meshes , 2012, ACM Trans. Graph..

[5]  D. Cohen-Or,et al.  Upright orientation of man-made objects , 2008, SIGGRAPH 2008.

[6]  Marc Sebban,et al.  Metric Learning , 2015, Metric Learning.

[7]  Kun Zhou,et al.  Mesh quilting for geometric texture synthesis , 2006, ACM Trans. Graph..

[8]  Siddhartha Chaudhuri,et al.  A probabilistic model for component-based shape synthesis , 2012, ACM Trans. Graph..

[9]  Daniel Cohen-Or,et al.  Feature-aligned shape texturing , 2009, ACM Trans. Graph..

[10]  G. Alecian,et al.  Diffusion in magnetic fields , 2004, Proceedings of the International Astronomical Union.

[11]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, ACM Trans. Graph..

[12]  Dani Lischinski,et al.  Layered shape synthesis: automatic generation of control maps for non-stationary textures , 2009, ACM Trans. Graph..

[13]  Daniel Cohen-Or,et al.  Upright orientation of man-made objects , 2008, ACM Trans. Graph..

[14]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[15]  H. Seidel,et al.  A connection between partial symmetry and inverse procedural modeling , 2010, ACM Trans. Graph..

[16]  Greg Turk,et al.  Geometric texture synthesis by example , 2004, SGP '04.

[17]  Dani Lischinski,et al.  Self Tuning Texture Optimization , 2015, Comput. Graph. Forum.

[18]  S. Helgason.  Differential Geometry, Lie Groups, and Symmetric Spaces , 1978 .

[19]  Rui Ma,et al.  Topology-varying 3D shape creation via structural blending , 2014, ACM Trans. Graph..

[20]  Dan B. Goldman,et al.  Non-parametric Texture Transfer Using MeshMatch , 2012 .

[21]  Wilmot Li,et al.  Style compatibility for 3D furniture models , 2015, ACM Trans. Graph..

[22]  Shai Kaspi.  High-resolution X-ray spectra of quasars , 2004, Proceedings of the International Astronomical Union.

[23]  Keenan Crane,et al.  Stripe patterns on surfaces , 2015, ACM Trans. Graph..

[24]  Marc Levoy,et al.  Texture synthesis over arbitrary manifold surfaces , 2001, SIGGRAPH.

[25]  Sylvain Lefebvre,et al.  State of the Art in Example-based Texture Synthesis , 2009, Eurographics.

[26]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[27]  Henning Biermann,et al.  Texture and Shape Synthesis on Surfaces , 2001, Rendering Techniques.

[28]  Vladimir G. Kim,et al.  Entropic metric alignment for correspondence problems , 2016, ACM Trans. Graph..

[29]  Kenneth Moreland,et al.  Diverging Color Maps for Scientific Visualization , 2009, ISVC.

[30]  Ralph R. Martin,et al.  Geometric texture synthesis and transfer via geometry images , 2005, SPM '05.

[31]  Leif Kobbelt,et al.  OpenFlipper: An Open Source Geometry Processing and Rendering Framework , 2010, Curves and Surfaces.

[32]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[33]  Alla Sheffer,et al.  Elements of style , 2015, ACM Trans. Graph..

[34]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, ACM Trans. Graph..

[35]  Alla Sheffer,et al.  Analogy‐driven 3D style transfer , 2014, Comput. Graph. Forum.

[36]  Daniel Cohen-Or,et al.  Consistent mesh partitioning and skeletonisation using the shape diameter function , 2008, The Visual Computer.

[37]  Brian Kulis,et al.  Metric Learning: A Survey , 2013, Found. Trends Mach. Learn..

[38]  Leonidas J. Guibas,et al.  Unsupervised texture transfer from images to model collections , 2016, ACM Trans. Graph..

[39]  Leif Kobbelt,et al.  A remeshing approach to multiresolution modeling , 2004, SGP '04.

[40]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[41]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[42]  Martin Reimers,et al.  Geodesic Polar Coordinates on Polygonal Meshes , 2012, Comput. Graph. Forum.

[43]  Jiawen Chen,et al.  Texture transfer using geometry correlation , 2006, EGSR '06.

[44]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[45]  Greg Turk,et al.  Texture synthesis on surfaces , 2001, SIGGRAPH.

[46]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[47]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[48]  James F. O'Brien,et al.  Example-based wrinkle synthesis for clothing animation , 2010, ACM Trans. Graph..

[49]  Olga Sorkine-Hornung,et al.  Integrable PolyVector fields , 2015, ACM Trans. Graph..

[50]  Baining Guo,et al.  Context-aware textures , 2007, TOGS.

[51]  Olga Sorkine-Hornung,et al.  Synthesis of Complex Image Appearance from Limited Exemplars , 2015, TOGS.

[52]  Leonidas J. Guibas,et al.  Near-Regular Structure Discovery Using Linear Programming , 2014, ACM Trans. Graph..

[53]  Stephen DiVerdi,et al.  Exploring collections of 3D models using fuzzy correspondences , 2012, ACM Trans. Graph..