Functionality preserving shape style transfer

When geometric models with a desired combination of style and functionality are not available, they currently need to be created manually. We facilitate algorithmic synthesis of 3D models of man-made shapes which combines user-specified style, described via an exemplar shape, and functionality, encoded by a functionally different target shape. Our method automatically transfers the style of the exemplar to the target, creating the desired combination. The main challenge in performing cross-functional style transfer is to implicitly separate an object's style from its function: while stylistically the output shapes should be as close as possible to the exemplar, their original functionality and structure, as encoded by the target, should be strictly preserved. Recent literature point to the presence of similarly shaped, salient geometric elements as a main indicator of stylistic similarity between 3D shapes. We therefore transfer the exemplar style to the target via a sequence of element-level operations. We allow only compatible operations, ones that do not affect the target functionality. To this end, we introduce a cross-structural element compatibility metric that estimates the impact of each operation on the edited shape. Our metric is based on the global context and coarse geometry of evaluated elements, and is trained on databases of 3D objects. We use this metric to cast style transfer as a tabu search, which incrementally updates the target shape using compatible operations, progressively increasing its style similarity to the exemplar while strictly maintaining its functionality at each step. We evaluate our framework across a range of man-made objects including furniture, light fixtures, and tableware, and perform a number of user studies confirming that it produces convincing outputs combining the desired style and function.

[1]  Leonidas J. Guibas,et al.  Shape2Pose , 2014, ACM Trans. Graph..

[2]  Yun Jiang,et al.  Hallucinated Humans as the Hidden Context for Labeling 3D Scenes , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Daniel Cohen-Or,et al.  SHED: shape edit distance for fine-grained shape similarity , 2015, ACM Trans. Graph..

[4]  Derek Nowrouzezahrai,et al.  Data-driven curvature for real-time line drawing of dynamic scenes , 2009, ACM Trans. Graph..

[5]  Levent Burak Kara,et al.  Co-abstraction of shape collections , 2012, ACM Trans. Graph..

[6]  Evangelos Kalogerakis,et al.  Eurographics Symposium on Geometry Processing 2015 Analysis and Synthesis of 3d Shape Families via Deep-learned Generative Models of Surfaces , 2022 .

[7]  Alla Sheffer,et al.  Analogy‐driven 3D style transfer , 2014, Comput. Graph. Forum.

[8]  Szymon Rusinkiewicz,et al.  Modeling by example , 2004, ACM Trans. Graph..

[9]  Vladlen Koltun,et al.  Single-view reconstruction via joint analysis of image and shape collections , 2015, ACM Trans. Graph..

[10]  Levent Burak Kara,et al.  Semantic shape editing using deformation handles , 2015, ACM Trans. Graph..

[11]  Leonidas J. Guibas,et al.  Fine-grained semi-supervised labeling of large shape collections , 2013, ACM Trans. Graph..

[12]  Ariel Shamir,et al.  Learning how objects function via co-analysis of interactions , 2016, ACM Trans. Graph..

[13]  D. Cohen-Or,et al.  Style-content separation by anisotropic part scales , 2010, ACM Trans. Graph..

[14]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[15]  Daniel Cohen-Or,et al.  Fit and diverse , 2012, ACM Trans. Graph..

[16]  WangRui,et al.  Analogy-driven 3D style transfer , 2014 .

[17]  Kun Zhou,et al.  Large mesh deformation using the volumetric graph Laplacian , 2005, ACM Trans. Graph..

[18]  WangRui,et al.  Functionality preserving shape style transfer , 2016 .

[19]  H. Seidel,et al.  A connection between partial symmetry and inverse procedural modeling , 2010, ACM Trans. Graph..

[20]  Alla Sheffer,et al.  Model Composition from Interchangeable Components , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[21]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[22]  Y. Wang,et al.  Curve Style Analysis in a Set of Shapes , 2013, Comput. Graph. Forum.

[23]  Siddhartha Chaudhuri,et al.  A probabilistic model for component-based shape synthesis , 2012, ACM Trans. Graph..

[24]  Daniel Cohen-Or,et al.  Smart Variations: Functional Substructures for Part Compatibility , 2013, Comput. Graph. Forum.

[25]  Hamid Laga,et al.  Geometry and context for semantic correspondences and functionality recognition in man-made 3D shapes , 2013, TOGS.

[26]  Ligang Liu,et al.  Interaction context (ICON) , 2015, ACM Trans. Graph..

[27]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[28]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[29]  Radomír Mech,et al.  Learning design patterns with bayesian grammar induction , 2012, UIST.

[30]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[31]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[32]  Alla Sheffer,et al.  Elements of style , 2015, ACM Trans. Graph..

[33]  Wilmot Li,et al.  Style compatibility for 3D furniture models , 2015, ACM Trans. Graph..

[34]  H. Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, SIGGRAPH 2004.

[35]  Li Fei-Fei,et al.  Reasoning about Object Affordances in a Knowledge Base Representation , 2014, ECCV.

[36]  Matthias Nießner,et al.  PiGraphs , 2016, ACM Trans. Graph..

[37]  Daniel Cohen-Or,et al.  Non-homogeneous resizing of complex models , 2008, SIGGRAPH Asia '08.

[38]  Miles Lewis Architectura : elements of architectural style , 2008 .

[39]  Niloy J. Mitra,et al.  Replaceable Substructures for Efficient Part‐Based Modeling , 2015, Comput. Graph. Forum.

[40]  Levent Burak Kara,et al.  Co-constrained handles for deformation in shape collections , 2014, ACM Trans. Graph..

[41]  Daniel Cohen-Or,et al.  Weak Convex Decomposition by Lines‐of‐sight , 2013, SGP '13.

[42]  Pat Hanrahan,et al.  SceneGrok: inferring action maps in 3D environments , 2014, ACM Trans. Graph..

[43]  Leonidas J. Guibas,et al.  Probabilistic reasoning for assembly-based 3D modeling , 2011, ACM Trans. Graph..

[44]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[45]  Manuel Laguna,et al.  Tabu Search , 1997 .

[46]  Nuria Oliver,et al.  Curve Analogies , 2002, Rendering Techniques.

[47]  Vladimir G. Kim,et al.  Data‐Driven Shape Analysis and Processing , 2015, Comput. Graph. Forum.

[48]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[49]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[50]  Adam Tauman Kalai,et al.  Adaptively Learning the Crowd Kernel , 2011, ICML.

[51]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.