Video synchronization and its application to object transfer

In video post-production applications, camera motion analysis and alignment are important in order to ensure the geometric correctness and temporal consistency. In this paper, we trade some generality in estimating and aligning camera motion for reduced computational complexity and increased image-based nature. The main contribution is to use fundamental ratios to synchronize video sequences of distinct scenes captured by cameras undergoing similar motions. We also present a simple method to align 3D camera trajectories when the fundamental ratios are not able to match the noisy trajectories. Experimental results show that our method can accurately synchronize sequences even when the scenes are totally different and have dense depths. An application on 3D object transfer is also demonstrated.

[1]  Patrick Bouthemy,et al.  A unified approach to shot change detection and camera motion characterization , 1999, IEEE Trans. Circuits Syst. Video Technol..

[2]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[3]  Lior Wolf,et al.  Sequence-to-Sequence Self Calibration , 2002, ECCV.

[4]  Michael Werman,et al.  The viewing graph , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Yaron Caspi,et al.  A step towards sequence-to-sequence alignment , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[7]  Luc Van Gool,et al.  Synchronizing video sequences , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Dan B. Goldman,et al.  Shadow Matting and Compositing , .

[9]  Kiriakos N. Kutulakos,et al.  Linear sequence-to-sequence alignment , 2004, CVPR 2004.

[10]  Henri Nicolas.  Shadow synthesis for video postproduction , 2005, IEEE Signal Processing Letters.

[11]  Gerhard Roth,et al.  Evolutionary Based Autocalibration from the Fundamental Matrix , 2002, EvoWorkshops.

[12]  Denis Simakov,et al.  Feature-Based Sequence-to-Sequence Matching , 2006, International Journal of Computer Vision.

[13]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[14]  Xiaochun Cao,et al.  Video Understanding and Content-Based Retrieval , 2005, TRECVID.

[15]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[16]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ian D. Reid,et al.  Synchronizing Image Sequences of Non-Rigid Objects , 2003, BMVC.

[19]  Paul F. Whelan,et al.  Projective rectification from the fundamental matrix , 2005, Image Vis. Comput..

[20]  Yaron Caspi,et al.  Alignment of non-overlapping sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[21]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[22]  Patrick Pérez,et al.  Periodic motion detection and segmentation via approximate sequence alignment , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[23]  Gideon P. Stein,et al.  Tracking from multiple view points: Self-calibration of space and time , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  T. Poggio,et al.  Synthesis and recognition of biological motion patterns based on linear superposition of prototypical motion sequences , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[25]  Xiaochun Cao,et al.  Single view compositing with shadows , 2005, The Visual Computer.

[26]  Seth J. Teller,et al.  Video matching , 2004, Encyclopedia of Multimedia.

[27]  Tanveer F. Syeda-Mahmood,et al.  View-invariant alignment and matching of video sequences , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[28]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[29]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[30]  Xiaochun Cao,et al.  3D Object Transfer Between Non-Overlapping Videos , 2006, IEEE Virtual Reality Conference (VR 2006).

[31]  Prosenjit Bose,et al.  Temporal Synchronization of Video Sequences in Theory and in Practice , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.