3D Mesh Segmentation via Multi-branch 1D Convolutional Neural Networks

There is an increasing interest in applying deep learning to 3D mesh segmentation. We observe that 1) existing feature-based techniques are often slow or sensitive to feature resizing, 2) there are minimal comparative studies and 3) techniques often suffer from reproducibility issue. This study contributes in two ways. First, we propose a novel convolutional neural network (CNN) for mesh segmentation. It uses 1D data, filters and a multi-branch architecture for separate training of multi-scale features. Together with a novel way of computing conformal factor (CF), our technique clearly out-performs existing work. Secondly, we publicly provide implementations of several deep learning techniques, namely, neural networks (NNs), autoencoders (AEs) and CNNs, whose architectures are at least two layers deep. The significance of this study is that it proposes a robust form of CF, offers a novel and accurate CNN technique, and a comprehensive study of several deep learning techniques for baseline comparison.

[1]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[2]  Craig Gotsman,et al.  Characterizing Shape Using Conformal Factors , 2008, 3DOR@Eurographics.

[3]  Ariel Shamir,et al.  A survey on Mesh Segmentation Techniques , 2008, Comput. Graph. Forum.

[4]  Ioannis Pratikakis,et al.  A comprehensive overview of methodologies and performance evaluation frameworks in 3D mesh segmentation , 2015, Comput. Vis. Image Underst..

[5]  Min Meng,et al.  Unsupervised co-segmentation for 3D shapes using iterative multi-label optimization , 2013, Comput. Aided Des..

[6]  Daniel Cohen-Or,et al.  Salient geometric features for partial shape matching and similarity , 2006, TOGS.

[7]  Daniel Cohen-Or,et al.  Contextual Part Analogies in 3D Objects , 2010, International Journal of Computer Vision.

[8]  Qinping Zhao,et al.  Image2Scene: Transforming Style of 3D Room , 2015, ACM Multimedia.

[9]  Ligang Liu,et al.  Co‐Segmentation of 3D Shapes via Subspace Clustering , 2012, Comput. Graph. Forum.

[10]  Daniel Cohen-Or,et al.  A Part‐aware Surface Metric for Shape Analysis , 2009, Comput. Graph. Forum.

[11]  Ayellet Tal,et al.  Metamorphosis of Polyhedral Surfaces using Decomposition , 2002, Comput. Graph. Forum.

[12]  Marco Attene,et al.  Hierarchical mesh segmentation based on fitting primitives , 2006, The Visual Computer.

[13]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[14]  Yu Zhang,et al.  Unsupervised 3D shape segmentation and co-segmentation via deep learning , 2016, Comput. Aided Geom. Des..

[15]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[16]  Ligang Liu,et al.  3D Shape Segmentation and Labeling via Extreme Learning Machine , 2014, Comput. Graph. Forum.

[17]  Yin Yang,et al.  Boundary-Aware Multidomain Subspace Deformation , 2013, IEEE Transactions on Visualization and Computer Graphics.

[18]  Subhransu Maji,et al.  3D Shape Segmentation with Projective Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Lin Wang,et al.  Garment modeling with a depth camera , 2015, ACM Trans. Graph..

[20]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[22]  Xiaowu Chen,et al.  3D Mesh Labeling via Deep Convolutional Neural Networks , 2015, ACM Trans. Graph..

[23]  Iasonas Kokkinos,et al.  Scale-invariant heat kernel signatures for non-rigid shape recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Ligang Liu,et al.  Projective Feature Learning for 3D Shapes with Multi‐View Depth Images , 2015, Comput. Graph. Forum.

[25]  Kun Zhou,et al.  Mesh editing with poisson-based gradient field manipulation , 2004, ACM Trans. Graph..

[26]  Daniel Cohen-Or,et al.  Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering , 2011, ACM Trans. Graph..

[27]  Aaron Hertzmann,et al.  Learning 3D mesh segmentation and labeling , 2010, ACM Trans. Graph..

[28]  Thomas A. Funkhouser,et al.  Randomized cuts for 3D mesh analysis , 2008, SIGGRAPH Asia '08.

[29]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[30]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[32]  Vladlen Koltun,et al.  Joint shape segmentation with linear programming , 2011, ACM Trans. Graph..

[33]  Guillaume Lavoué,et al.  Learning Boundary Edges for 3D‐Mesh Segmentation , 2011, Comput. Graph. Forum.

[34]  Baoquan Chen,et al.  Unsupervised co-segmentation of 3D shapes via affinity aggregation spectral clustering , 2013, Comput. Graph..

[35]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[36]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[37]  Daniel Cohen-Or,et al.  Consistent mesh partitioning and skeletonisation using the shape diameter function , 2008, The Visual Computer.

[38]  Thomas A. Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, ACM Trans. Graph..

[39]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[40]  Daniel Cohen-Or,et al.  SHED: shape edit distance for fine-grained shape similarity , 2015, ACM Trans. Graph..

[41]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[42]  Ersin Yumer,et al.  Convolutional neural networks on surfaces via seamless toric covers , 2017, ACM Trans. Graph..

[43]  Yang Liu,et al.  O-CNN , 2017, ACM Trans. Graph..

[45]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Ligang Liu,et al.  Interaction context (ICON) , 2015, ACM Trans. Graph..

[47]  Daniel Cohen-Or,et al.  Shape Segmentation by Approximate Convexity Analysis , 2014, ACM Trans. Graph..