Performing Image-like Convolution on Triangular Meshes

Image convolution with a filtering mask is at the base of several image analysis operations. This is motivated by Mathematical foundations and by the straightforward way the discrete convolution can be computed on a grid-like domain. Extending the convolution operation to the mesh manifold support is a challenging task due to the irregular structure of the mesh connections. In this paper, we propose a computational framework that allows convolutional operations on the mesh. This relies on the idea of ordering the facets of the mesh so that a shift-like operation can be derived. Experiments have been performed with several filter masks (Sobel, Gabor, etc.) showing state-of-the-art results in 3D relief patterns retrieval on the SHREC'17 dataset. We also provide evidence that the proposed framework can enable convolution and pooling-like operations as can be needed for extending Convolutional Neural Networks to 3D meshes.

[1]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[2]  Yang Liu,et al.  O-CNN , 2017, ACM Trans. Graph..

[3]  Naoufel Werghi,et al.  An ordered topological representation of 3D triangular mesh facial surface: concept and applications , 2012, EURASIP J. Adv. Signal Process..

[4]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[5]  Hae-Jeong Park,et al.  Geometric Convolutional Neural Network for Analyzing Surface-Based Neuroimaging Data , 2018, Front. Neuroinform..

[6]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[7]  Andrea Giachetti,et al.  Shrec'17 Track: Retrieval of surfaces with similar relief patterns , 2017 .

[8]  Alberto Del Bimbo,et al.  Local binary patterns on triangular meshes: Concept and applications , 2015, Comput. Vis. Image Underst..

[9]  Edward K. Wong,et al.  Deepshape: Deep learned shape descriptor for 3D shape matching and retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[12]  Alberto Del Bimbo,et al.  Computing Local Binary Patterns on Mesh Manifolds for 3D Texture Retrieval , 2015, 3DOR@Eurographics.

[13]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[14]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[16]  Karthik Ramani,et al.  Deep Learning 3D Shape Surfaces Using Geometry Images , 2016, ECCV.

[17]  Pierre Vandergheynst,et al.  Learning class‐specific descriptors for deformable shapes using localized spectral convolutional networks , 2015, SGP '15.