Subdivision-Based Mesh Convolution Networks

Fig. 1. SubdivNet, a subdivision-based mesh convolution network for deep geometric learning. Given a mesh as input, we construct a hierarchical subdivision structure with a pyramid of regular connectivities, analogous to a 2D image pyramid. This structure permits natural notions of convolution, pooling, and upsampling operation on 3D meshes, which together provide the building blocks of our mesh-based deep neural network. Our network is effective and efficient for mesh-based representation learning in a variety of applications.

[1]  Yue Gao,et al.  MeshNet: Mesh Neural Network for 3D Shape Representation , 2018, AAAI.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Wei Liu,et al.  Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images , 2018, ECCV.

[4]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[6]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Alexander M. Bronstein,et al.  Deep Functional Maps: Structured Prediction for Dense Shape Correspondence , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Malcolm A. Sabin,et al.  Behaviour of recursive division surfaces near extraordinary points , 1998 .

[10]  Vladlen Koltun,et al.  Tangent Convolutions for Dense Prediction in 3D , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[12]  Ersin Yumer,et al.  Convolutional neural networks on surfaces via seamless toric covers , 2017, ACM Trans. Graph..

[13]  Xianzhi Li,et al.  DNF-Net: A Deep Normal Filtering Network for Mesh Denoising , 2020, IEEE Transactions on Visualization and Computer Graphics.

[14]  Andrei Khodakovsky,et al.  Hybrid meshes: multiresolution using regular and irregular refinement , 2002, SCG '02.

[15]  Leif Kobbelt,et al.  √3-subdivision , 2000, SIGGRAPH.

[16]  Dragomir Anguelov,et al.  SCAPE: shape completion and animation of people , 2005, ACM Trans. Graph..

[17]  Michael J. Black,et al.  Generating 3D faces using Convolutional Mesh Autoencoders , 2018, ECCV.

[18]  Maks Ovsjanikov,et al.  Multi-directional geodesic neural networks via equivariant convolution , 2018, ACM Trans. Graph..

[19]  N. Dyn,et al.  A butterfly subdivision scheme for surface interpolation with tension control , 1990, TOGS.

[20]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[21]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[23]  Davide Scaramuzza,et al.  Primal-Dual Mesh Convolutional Neural Networks , 2020, NeurIPS.

[24]  Wei Wu,et al.  PointCNN: Convolution On X-Transformed Points , 2018, NeurIPS.

[25]  Abhishek Sharma,et al.  Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Gershon Elber,et al.  Orientation analysis of 3D objects toward minimal support volume in 3D-printing , 2015, Comput. Graph..

[27]  Victor S. Lempitsky,et al.  Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Lin Gao,et al.  A survey on deep geometry learning: From a representation perspective , 2020, Computational Visual Media.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Vladimir G. Kim,et al.  Neural subdivision , 2020, ACM Trans. Graph..

[31]  Zhichao Zhou,et al.  DeepPano: Deep Panoramic Representation for 3-D Shape Recognition , 2015, IEEE Signal Processing Letters.

[32]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[33]  Ming Dong,et al.  Directionally Convolutional Networks for 3D Shape Segmentation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Stephan Günnemann,et al.  Dual-Primal Graph Convolutional Networks , 2018, ArXiv.

[35]  Andrei Khodakovsky,et al.  Globally smooth parameterizations with low distortion , 2003, ACM Trans. Graph..

[36]  Ralph R. Martin,et al.  PCT: Point cloud transformer , 2020, Computational Visual Media.

[37]  Shi-Min Hu,et al.  Jittor: a novel deep learning framework with meta-operators and unified graph execution , 2020, Science China Information Sciences.

[38]  Charles T. Loop,et al.  Smooth Subdivision Surfaces Based on Triangles , 1987 .

[39]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[40]  Ron Goldman,et al.  Nonlinear subdivision through nonlinear averaging , 2008, Comput. Aided Geom. Des..

[41]  Vladimir G. Kim,et al.  GWCNN: A Metric Alignment Layer for Deep Shape Analysis , 2017, Comput. Graph. Forum.

[42]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[43]  Maks Ovsjanikov,et al.  Unsupervised Deep Learning for Structured Shape Matching , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Ilya Kostrikov,et al.  Surface Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Stefanos Zafeiriou,et al.  SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[46]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[47]  E. Catmull,et al.  Recursively generated B-spline surfaces on arbitrary topological meshes , 1978 .

[48]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[49]  Leonidas J. Guibas,et al.  SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Jing Ren,et al.  Continuous and orientation-preserving correspondences via functional maps , 2018, ACM Trans. Graph..

[51]  Bastian Leibe,et al.  DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Hans-Peter Seidel,et al.  VNect , 2017, ACM Trans. Graph..

[53]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Paul Suetens,et al.  SHREC '11 Track: Shape Retrieval on Non-rigid 3D Watertight Meshes , 2011, 3DOR@Eurographics.

[55]  Jing Ren,et al.  ZoomOut: Spectral Upsampling for Efficient Shape Correspondence , 2019, ACM Trans. Graph..

[56]  Johannes Wallner,et al.  Geometric Modeling with Conical Meshes and Developable Surfaces , 2006, ACM Trans. Graph..

[57]  Jieyu Zhao,et al.  Mesh Convolution: A Novel Feature Extraction Method for 3D Nonrigid Object Classification , 2021, IEEE Transactions on Multimedia.

[58]  Yaron Lipman,et al.  Surface Networks via General Covers , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[59]  Justin Solomon,et al.  HodgeNet , 2021, ACM Trans. Graph..

[60]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[61]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Ayellet Tal,et al.  MeshWalker: Deep Mesh Understanding by Random Walks , 2020, ACM Trans. Graph..

[63]  Karthik Ramani,et al.  Deep Learning 3D Shape Surfaces Using Geometry Images , 2016, ECCV.

[64]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Keenan Crane,et al.  Navigating intrinsic triangulations , 2019, ACM Trans. Graph..

[66]  Leonidas J. Guibas,et al.  TextureNet: Consistent Local Parametrizations for Learning From High-Resolution Signals on Meshes , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Marcel Campen,et al.  A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes , 2018, ECCV Workshops.

[68]  Li Wang,et al.  MeshSNet: Deep Multi-scale Mesh Feature Learning for End-to-End Tooth Labeling on 3D Dental Surfaces , 2019, MICCAI.

[69]  Leonidas J. Guibas,et al.  Robust Watertight Manifold Surface Generation Method for ShapeNet Models , 2018, ArXiv.

[70]  Xin Tong,et al.  PFCNN: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames , 2018, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Shi-Min Hu,et al.  High-Quality Textured 3D Shape Reconstruction with Cascaded Fully Convolutional Networks , 2021, IEEE Transactions on Visualization and Computer Graphics.

[72]  Matthias Nießner,et al.  Scan2Mesh: From Unstructured Range Scans to 3D Meshes , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[74]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[75]  Hans-Peter Seidel,et al.  A Shrink Wrapping Approach to Remeshing Polygonal Surfaces , 1999, Comput. Graph. Forum.

[76]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Mathieu Aubry,et al.  3D-CODED: 3D Correspondences by Deep Deformation , 2018, ECCV.

[78]  Peter Schröder,et al.  Normal meshes , 2000, SIGGRAPH.

[79]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[80]  Daniel Cohen-Or,et al.  Active co-analysis of a set of shapes , 2012, ACM Trans. Graph..

[81]  Jonathan Masci,et al.  Learning shape correspondence with anisotropic convolutional neural networks , 2016, NIPS.

[82]  Daniela Giorgi,et al.  SHape REtrieval Contest 2007: Watertight Models Track , 2007 .

[83]  David P. Dobkin,et al.  MAPS: multiresolution adaptive parameterization of surfaces , 1998, SIGGRAPH.