Formalization of the General Video Temporal Synchronization Problem

In this work, we present a theoretical formalization of the temporal synchronization problem and a method to temporally synchronize multiple stationary video cameras with overlapping views of the same scene. The method uses a two stage approach that first approximates the synchronization by tracking moving objects and identifying curvature points. The method then proceeds to refine the estimate using a consensus based matching heuristic to find frames that best agree with the pre-computed camera geometries from stationary background image features. By using the fundamental matrix and the trifocal tensor in the second refinement step, we improve the estimation of the first step and handle a broader more generic range of input scenarios and camera conditions. The method is relatively simple compared to current techniques and is no harder than feature tracking in stage one and computing accurate geometries in stage two. We also provide a robust method to assist synchronization in the presence of inaccurate geometry computation, and a theoretical limit on the accuracy that can be expected from any synchronization system.

[1]  Seth J. Teller,et al.  Video matching , 2004, Encyclopedia of Multimedia.

[2]  Yaron Caspi,et al.  Alignment of non-overlapping sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Patrick Pérez,et al.  Periodic motion detection and segmentation via approximate sequence alignment , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Gang Xu,et al.  Epipolar Geometry in Stereo, Motion and Object Recognition , 1996, Computational Imaging and Vision.

[5]  C Tomasi,et al.  Shape and motion from image streams: a factorization method. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[6]  C. V. Jawahar,et al.  Video frame alignment in multiple views , 2002, Proceedings. International Conference on Image Processing.

[7]  Denis Simakov,et al.  Feature-Based Sequence-to-Sequence Matching , 2006, International Journal of Computer Vision.

[8]  Marc Pollefeys,et al.  Synchronization and calibration of camera networks from silhouettes , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[9]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Ian D. Reid,et al.  Synchronizing Image Sequences of Non-Rigid Objects , 2003, BMVC.

[11]  Yi-Ping Hung,et al.  Disparity-based view interpolation for multiple-perspective stereoscopic displays , 2000, Electronic Imaging.

[12]  Andrew Zisserman,et al.  Robust parameterization and computation of the trifocal tensor , 1997, Image Vis. Comput..

[13]  Luc Van Gool,et al.  Synchronizing video sequences , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[14]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[15]  Kiriakos N. Kutulakos,et al.  Linear Sequence-to-Sequence Alignment , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Tanveer F. Syeda-Mahmood,et al.  View-invariant alignment and matching of video sequences , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Denis Simakov,et al.  Space-time scene manifolds , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  Anthony Whitehead,et al.  The Projective Vision Toolkit , 2001 .

[19]  Isaac Cohen,et al.  Continuous multi-views tracking using tensor voting , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[20]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.