Subframe Video Synchronization via 3D Phase Correlation

This paper introduces an accurate approach for synchronization (temporal alignment) between two video sequences of the same dynamic scene captured by uncalibrated cameras. With the homography assumption in spatial domain, an iterative procedure that successively achieves the alignment in space and time is proposed and its convergence is experimentally verified. Subframe accuracy is achieved by extending the existing image subpixel registration scheme to subframe video synchronization. In order to demonstrate the accuracy of the proposed method, we adopt a novel use of audio signals for their high sampling rate to obtain the synchronization ground-truth. The proposed video synchronization technique has potential use in temporal super-resolution, image-based rendering and tele-immersion.

[1]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Kiriakos N. Kutulakos,et al.  Linear Sequence-to-Sequence Alignment , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[4]  Takeshi Naemura,et al.  Real-Time Video-Based Modeling and Rendering of 3D Scenes , 2002, IEEE Computer Graphics and Applications.

[5]  Denis Simakov,et al.  Feature-Based Sequence-to-Sequence Matching , 2006, International Journal of Computer Vision.

[6]  Tanveer F. Syeda-Mahmood,et al.  View-invariant alignment and matching of video sequences , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Hassan Foroosh,et al.  Extension of phase correlation to subpixel registration , 2002, IEEE Trans. Image Process..

[8]  Gideon P. Stein,et al.  Tracking from multiple view points: Self-calibration of space and time , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Yaron Caspi,et al.  Aligning Non-Overlapping Sequences , 2004, International Journal of Computer Vision.

[10]  Yaron Caspi,et al.  Under the supervision of , 2003 .

[11]  Yaron Caspi,et al.  A step towards sequence-to-sequence alignment , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Takeo Kanade,et al.  Spatio-Temporal View Interpolation , 2002, Rendering Techniques.

[13]  Ian D. Reid,et al.  Goal-directed Video Metrology , 1996, ECCV.

[14]  Yaron Caspi,et al.  Alignment of non-overlapping sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).