Segmentation of temporal mesh sequences into rigidly moving components

In this paper is considered the segmentation of meshes into rigid components given temporal sequences of deforming meshes. We propose a fully automatic approach that identifies model parts that consistently move rigidly over time. This approach can handle meshes independently reconstructed at each time instant. It allows therefore for sequences of meshes with varying connectivities as well as varying topology. It incrementally adapts, merges and splits segments along a sequence based on the coherence of motion information within each segment. In order to provide tools for the evaluation of the approach, we also introduce new criteria to quantify a mesh segmentation. Results on both synthetic and real data as well as comparisons are provided in the paper.

[1]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[2]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[3]  Luiz Velho,et al.  A Hierarchical Segmentation of Articulated Bodies , 2008, Comput. Graph. Forum.

[4]  Hao Zhang,et al.  Mesh Segmentation via Spectral Embedding and Contour Analysis , 2007, Comput. Graph. Forum.

[5]  Yücel Yemez,et al.  3D Articulated Shape Segmentation Using Motion Information , 2010, 2010 20th International Conference on Pattern Recognition.

[6]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[7]  Leonidas J. Guibas,et al.  Shape Decomposition using Modal Analysis , 2009, Comput. Graph. Forum.

[8]  Kiyoharu Aizawa,et al.  Hierarchical mesh decomposition and motion tracking for Time-Varying-Meshes , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[9]  Takashi Matsuyama,et al.  3D video performance segmentation , 2010, 2010 IEEE International Conference on Image Processing.

[10]  Kiyoharu Aizawa,et al.  Motion Segmentation and Retrieval for 3D Video Based on Modified Shape Distribution , 2007, EURASIP J. Adv. Signal Process..

[11]  Roberto Scopigno,et al.  Motion-based mesh segmentation using augmented silhouettes , 2012, Graph. Model..

[12]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[14]  Marco Attene,et al.  Hierarchical mesh segmentation based on fitting primitives , 2006, The Visual Computer.

[15]  Tong-Yee Lee,et al.  Segmenting a deforming mesh into near-rigid components , 2006, The Visual Computer.

[16]  Edmond Boyer,et al.  Learning temporally consistent rigidities , 2011, CVPR 2011.

[17]  Christine Depraz,et al.  Harmonic skeleton for realistic character animation , 2007, SCA '07.

[18]  Jed Lengyel,et al.  Compression of time-dependent geometry , 1999, SI3D.

[19]  Franck Hétroy,et al.  A Framework for motion-based mesh sequence segmentation , 2010 .

[20]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[21]  Edmond Boyer,et al.  Temporally Coherent Segmentation of 3D Reconstructions , 2010 .

[22]  Slobodan Ilic,et al.  Iterative Deformable Surface Tracking in Multi-View Setups , 2010 .

[23]  Michael Garland,et al.  Editing arbitrarily deforming surface animations , 2006, SIGGRAPH 2006.

[24]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[25]  Thomas J. Cashman,et al.  A continuous, editable representation for deforming mesh sequences with separate signals for time, pose and shape , 2012, Comput. Graph. Forum.

[26]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[27]  Takashi Matsuyama,et al.  Topology dictionary with Markov model for 3D video content-based skimming and description , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Alon Wolf,et al.  Group-Valued Regularization Framework for Motion Segmentation of Dynamic Non-rigid Shapes , 2011, SSVM.

[29]  Hao Zhang,et al.  Segmentation of 3D meshes through spectral clustering , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[30]  Alan Brunton,et al.  Segmenting animated objects into near-rigid components , 2010, The Visual Computer.

[31]  Florin Cutzu Computing 3D object parts from similarities among object views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[32]  Aaron Hertzmann,et al.  Learning 3D mesh segmentation and labeling , 2010, ACM Trans. Graph..

[33]  Thomas A. Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, ACM Trans. Graph..

[34]  Radu Horaud,et al.  Coherent Laplacian 3-D protrusion segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Karthik Ramani,et al.  Heat-mapping: A robust approach toward perceptually consistent mesh segmentation , 2011, CVPR 2011.

[36]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[37]  Daniel Cohen-Or,et al.  Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering , 2011, ACM Trans. Graph..

[38]  Radu Horaud,et al.  Learning Shape Segmentation Using Constrained Spectral Clustering and Probabilistic Label Transfer , 2010, ECCV.

[39]  Ariel Shamir,et al.  A survey on Mesh Segmentation Techniques , 2008, Comput. Graph. Forum.

[40]  Leonidas J. Guibas,et al.  Persistence-based segmentation of deformable shapes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[41]  Peter Grant,et al.  Proceedings IEEE International Conference on Image Processing (ICIP) , 1996 .

[42]  Martin Reuter,et al.  Hierarchical Shape Segmentation and Registration via Topological Features of Laplace-Beltrami Eigenfunctions , 2010, International Journal of Computer Vision.

[43]  Hans-Peter Seidel,et al.  Automatic Conversion of Mesh Animations into Skeleton‐based Animations , 2008, Comput. Graph. Forum.

[44]  Guillaume Lavoué,et al.  A framework for the objective evaluation of segmentation algorithms using a ground-truth of human segmented 3D-models , 2009, 2009 IEEE International Conference on Shape Modeling and Applications.

[45]  Eitan Grinspun,et al.  To appear in the ACM SIGGRAPH conference proceedings Efficient Simulation of Inextensible Cloth , 2007 .