Augmented Motion History Volume for Spatiotemporal Editing of 3-D Video in Multiparty Interaction Scenes

We present a novel method that performs spatio temporal editing of 3-D multiparty interaction scenes for free-viewpoint browsing, from separately captured 3-D video data. The main idea is to first propose the augmented motion history volume (aMHV) for individual motion representation. Then, by modeling the correlations between different aMHVs, we can define a multiparty interaction dictionary, describing the spatiotemporal constraints for different types of multi-party interaction events. Finally, a constraint satisfaction and a global optimization method synthesize natural and continuous 3-D multiparty interaction scenes. Evaluations with real data demonstrate the effectiveness of our method.

[1]  Adrian Hilton,et al.  Spherical matching for temporal correspondence of non-rigid surfaces , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[2]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[3]  Xiaojun Wu,et al.  Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Tomohiko Mukai,et al.  Geostatistical motion interpolation , 2005, SIGGRAPH 2005.

[5]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[6]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[7]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[8]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[9]  Jan Kautz,et al.  Video-based characters: creating new human performances from a multi-view video database , 2011, SIGGRAPH 2011.

[10]  Radu Horaud,et al.  Temporal Surface Tracking Using Mesh Evolution , 2008, ECCV.

[11]  Takashi Matsuyama,et al.  Augmented Motion History Volume for Spatiotemporal Editing of 3-D Video in Multiparty Interaction Scenes , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[13]  Michael Gleicher,et al.  Motion editing with spacetime constraints , 1997, SI3D.

[14]  Michael Gleicher,et al.  Parametric motion graphs , 2007, SI3D.

[15]  T. Matsuyama,et al.  Dynamic 3D shape from multi-viewpoint images using deformable mesh model , 2003, 3rd International Symposium on Image and Signal Processing and Analysis, 2003. ISPA 2003. Proceedings of the.

[16]  Christian Rössl,et al.  Dense correspondence finding for parametrization-free animation reconstruction from video , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Michael Gleicher,et al.  Automated extraction and parameterization of motions in large data sets , 2004, SIGGRAPH 2004.

[18]  Atsushi Nakazawa,et al.  Human video textures , 2009, I3D '09.

[19]  Okan Arikan,et al.  Interactive motion generation from examples , 2002, ACM Trans. Graph..

[20]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH Classes.

[21]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[22]  Slobodan Ilic,et al.  Free-form mesh tracking: A patch-based approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  James K. Hahn,et al.  Interpolation synthesis for articulated figure motion , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[24]  Adrian Hilton,et al.  Human motion synthesis from 3D video , 2009, CVPR.

[25]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[26]  Takashi Matsuyama,et al.  Topology Dictionary for 3D Video Understanding , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Sung Yong Shin,et al.  A hierarchical approach to interactive motion editing for human-like figures , 1999, SIGGRAPH.

[28]  Takashi Matsuyama,et al.  3D Face Reconstruction and Gaze Estimation from Multi-view Video using Symmetry Prior , 2012, IPSJ Trans. Comput. Vis. Appl..

[29]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[30]  Jean-Yves Guillemaut,et al.  Interactive Animation of 4D Performance Capture , 2013, IEEE Transactions on Visualization and Computer Graphics.

[31]  Adrian Hilton,et al.  Video-based character animation , 2005, SCA '05.

[32]  C. Karen Liu,et al.  Synthesis of Responsive Motion Using a Dynamic Model , 2010, Comput. Graph. Forum.

[33]  Victor B. Zordan,et al.  Dynamic response for motion capture animation , 2005, SIGGRAPH 2005.

[34]  Adrian Hilton,et al.  Correspondence labelling for wide-timeframe free-form surface matching , 2007, 2007 IEEE 11th International Conference on Computer Vision.