Hierarchical Shape Matching for Temporally Consistent 3D Video

In this paper we present a novel approach for temporal alignment of reconstructed mesh sequences with non-rigid surfaces to obtain a consistent representation. We propose a hierarchical scheme for non-sequential matching of frames across the sequence using shape similarity. This gives a tree structure which represents the optimal path for alignment of each frame in the sequence to minimize the change in shape. Non-rigid alignment is performed by recursively traversing the tree to align all frames. Non-sequential alignment reduces problems of drift or tracking failure which occur in previous sequential frame-to-frame techniques. Comparative evaluation on challenging 3D video sequences demonstrates that the proposed approach produces a temporally coherent representation with reduced error in shape and correspondence.

[1]  Slobodan Ilic,et al.  Iterative mesh deformation for dense surface tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[2]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[3]  Edmond Boyer,et al.  Visual Shapes of Silhouette Sets , 2006, 3DPVT.

[4]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[5]  Slobodan Ilic,et al.  Probabilistic Deformable Surface Tracking from Multiple Videos , 2010, ECCV.

[6]  Christian Rössl,et al.  Dense correspondence finding for parametrization-free animation reconstruction from video , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  Adrian Hilton,et al.  Correspondence labelling for wide-timeframe free-form surface matching , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[11]  Takashi Matsuyama,et al.  Dynamic surface matching by geodesic mapping for 3D animation transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[13]  Adrian Hilton,et al.  Shape Similarity for 3D Video Sequences of People , 2010, International Journal of Computer Vision.

[14]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Slobodan Ilic,et al.  Free-form mesh tracking: A patch-based approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Hans-Peter Seidel,et al.  A volumetric approach to interactive shape editing , 2007 .

[17]  Radu Horaud,et al.  Temporal Surface Tracking Using Mesh Evolution , 2008, ECCV.

[18]  Adrian Hilton,et al.  Spherical matching for temporal correspondence of non-rigid surfaces , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[19]  Tony Tung,et al.  Comparison of Skeleton and Non-Skeleton Shape Descriptors for 3D Video , 2010 .

[20]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[21]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.