UVStyle-Net: Unsupervised Few-shot Learning of 3D Style Similarity Measure for B-Reps

Boundary Representations (B-Reps) are the industry standard in 3D Computer Aided Design/Manufacturing (CAD/CAM) and industrial design due to their fidelity in representing stylistic details. However, they have been ignored in the 3D style research. Existing 3D style metrics typically operate on meshes or point clouds, and fail to account for end-user subjectivity by adopting fixed definitions of style, either through crowd-sourcing for style labels or hand-crafted features. We propose UVStyle-Net, a style similarity measure for B-Reps that leverages the style signals in the second order statistics of the activations in a pre-trained (unsupervised) 3D encoder, and learns their relative importance to a subjective end-user through few-shot learning. Our approach differs from all existing data-driven 3D style methods since it may be used in completely unsupervised settings, which is desirable given the lack of publicly available labeled B-Rep datasets. More importantly, the fewshot learning accounts for the inherent subjectivity associated with style. We show quantitatively that our proposed method with B-Reps is able to capture stronger style signals than alternative methods on meshes and point clouds despite its significantly greater computational efficiency. We also show it is able to generate meaningful style gradients with respect to the input shape, and that few-shot learning with as few as two positive examples selected by an enduser is sufficient to significantly improve the style measure. Finally, we demonstrate its efficacy on a large unlabeled public dataset of CAD models. Source code and data are available at github.com/AutodeskAILab/UVStyle-Net.

[1]  Roland Siegwart,et al.  3DSNet: Unsupervised Shape-to-Shape 3D Style Transfer , 2020, ArXiv.

[2]  Mei-Ling Shyu,et al.  A Survey on Deep Learning , 2018, ACM Comput. Surv..

[3]  Vladimir G. Kim,et al.  Neural subdivision , 2020, ACM Trans. Graph..

[4]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[6]  Kunwoo Lee,et al.  Partial Entity Structure: A Compact Boundary Representation for Non-Manifold Geometric Modeling , 2001, J. Comput. Inf. Sci. Eng..

[7]  Yun Fu,et al.  Multimodal Style Transfer via Graph Cuts , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Wilmot Li,et al.  Style compatibility for 3D furniture models , 2015, ACM Trans. Graph..

[9]  Trevor Darrell,et al.  Multi-content GAN for Few-Shot Font Style Transfer , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Mohammed Bennamoun,et al.  Deep Learning for 3D Point Clouds: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Alla Sheffer,et al.  Elements of Style: Learning Perceptual Shape Style Similarity , 2015 .

[14]  Karthik Ramani,et al.  3D Object Classification via Spherical Projections , 2017, 2017 International Conference on 3D Vision (3DV).

[15]  Yoshua Bengio,et al.  Understanding intermediate layers using linear classifier probes , 2016, ICLR.

[16]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[19]  Tse-Yu Pan,et al.  Deep model style: Cross-class style compatibility for 3D furniture within a scene , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[20]  Marc Alexa,et al.  ABC: A Big CAD Model Dataset for Geometric Deep Learning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jan Boehm,et al.  A review on deep learning techniques for 3D sensed data classification , 2019, Remote. Sens..

[22]  Max Welling,et al.  Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs , 2020, ICLR.

[23]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[24]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[25]  Jaakko Lehtinen,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Andrea Vedaldi,et al.  Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Leif Kobbelt,et al.  Identifying Style of 3D Shapes using Deep Metric Learning , 2016, Comput. Graph. Forum.

[28]  Kevin Weiler Topological Structures for Geometric Modeling , 1986 .

[29]  Xiang Pan,et al.  3D Patch-Based Sparse Learning for Style Feature Extraction , 2019, IEEE Access.

[30]  Gregory Cohen,et al.  EMNIST: an extension of MNIST to handwritten letters , 2017, CVPR 2017.

[31]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[32]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[33]  Chunxiao Liu,et al.  TSIT: A Simple and Versatile Framework for Image-to-Image Translation , 2020, ECCV.

[34]  Katashi Nagao,et al.  PSNet: A Style Transfer Network for Point Cloud Stylization on Geometry and Color , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[35]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..