Efficient feature tracking of time-varying surfaces using multi-scale motion flow propagation

This paper presents a framework for efficient feature tracking of time-varying surfaces. The framework can not only capture the dynamic geometry features on time-varying surfaces, but can also compute the accurate boundaries of the geometry features. The basic idea of the proposed approach is using the multi-scale motion flow and surface matching information to propagate the feature frame on time-varying surfaces. We first define an effective multi-scale geometry motion flow for the time-varying surfaces, which efficiently propagates the geometry features along the time direction of the time-varying surfaces. By combining both the approximately invariant signature vectors and geometry motion flow vectors, we also incorporate the shape matching into the system to process feature tracking for time-varying surfaces in large deformation while with low frame sampling rate. Our approach does not depend on the topological connection of the underlying surfaces. Thus, it can process both mesh-based and point-based time-varying surfaces without vertex-to-vertex correspondence across the frames. Feature tracking results on different kinds of time-varying surfaces illustrate the efficiency and effectiveness of the proposed method.

[1]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[2]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Hao Li,et al.  Tracking surfaces with evolving topology , 2012, ACM Trans. Graph..

[4]  Derek Bradley,et al.  High resolution passive facial performance capture , 2010, ACM Trans. Graph..

[5]  Pieter Peers,et al.  Temporally coherent completion of dynamic shapes , 2012, TOGS.

[6]  Meng Liu,et al.  Fast hierarchical animated object decomposition using approximately invariant signature , 2012, The Visual Computer.

[7]  Derek Bradley,et al.  Markerless garment capture , 2008, SIGGRAPH 2008.

[8]  Joachim Weickert,et al.  Combining the Advantages of Local and Global Optic Flow Methods , 2002, DAGM-Symposium.

[9]  Axel Pinz,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[10]  Leonidas J. Guibas,et al.  Eurographics Symposium on Geometry Processing (2007) Reconstruction of Deforming Geometry from Time-varying Point Clouds , 2022 .

[11]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[12]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[13]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[14]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.

[15]  Hans-Peter Seidel,et al.  Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data , 2009, TOGS.

[16]  Daniel Cohen-Or,et al.  Bilateral mesh denoising , 2003 .

[17]  Mathieu Desbrun,et al.  Discrete shells , 2003, SCA '03.

[18]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[19]  Yongwei Nie,et al.  Efficient Edit Propagation Using Hierarchical Data Structure , 2011, IEEE Transactions on Visualization and Computer Graphics.

[20]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[21]  Kun Zhou,et al.  Gradient domain editing of deforming mesh sequences , 2007, ACM Trans. Graph..

[22]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[23]  Leonidas J. Guibas,et al.  Dynamic geometry registration , 2007, Symposium on Geometry Processing.

[24]  Edmond Boyer,et al.  Learning temporally consistent rigidities , 2011, CVPR 2011.

[25]  Ronald Fedkiw,et al.  Animation and rendering of complex water surfaces , 2002, ACM Trans. Graph..

[26]  FuaPascal,et al.  Surface Deformation Models for Nonrigid 3D Shape Recovery , 2007 .

[27]  Hans-Peter Seidel,et al.  Motion capture using joint skeleton tracking and surface estimation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Ligang Liu,et al.  Sketch-based mesh cutting: A comparative study , 2012, Graph. Model..

[29]  Michael Garland,et al.  Editing arbitrarily deforming surface animations , 2006, ACM Trans. Graph..

[30]  Pascal Fua,et al.  Surface Deformation Models for Nonrigid 3D Shape Recovery , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Igor Guskov,et al.  Multi-scale features for approximate alignment of point-based surfaces , 2005, SGP '05.

[32]  Hans-Peter Seidel,et al.  Lightweight binocular facial performance capture under uncontrolled lighting , 2012, ACM Trans. Graph..

[33]  Ayellet Tal,et al.  Mesh segmentation using feature point and core extraction , 2005, The Visual Computer.

[34]  Daniel Cohen-Or,et al.  Contextual Part Analogies in 3D Objects , 2010, International Journal of Computer Vision.

[35]  M. Gross,et al.  Deforming meshes that split and merge , 2009, SIGGRAPH 2009.

[36]  Jing Xiao,et al.  Vision-based control of 3D facial animation , 2003, SCA '03.

[37]  Alan Brunton,et al.  Segmenting animated objects into near-rigid components , 2010, The Visual Computer.

[38]  Slobodan Ilic,et al.  Free-form mesh tracking: A patch-based approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[40]  Chunxia Xiao,et al.  Hierarchical aggregation for efficient shape extraction , 2009, The Visual Computer.

[41]  Tong-Yee Lee,et al.  Segmenting a deforming mesh into near-rigid components , 2006, The Visual Computer.

[42]  Hans-Peter Seidel,et al.  Marker-Less 3D Feature Tracking for Mesh-Based Human Motion Capture , 2007, Workshop on Human Motion.

[43]  Derek Bradley,et al.  High-quality passive facial performance capture using anchor frames , 2011, ACM Trans. Graph..

[44]  Dan A. Alcantara,et al.  Space-time surface reconstruction using incompressible flow , 2008, SIGGRAPH 2008.

[45]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.