Rolling Shutter Camera Synchronization with Sub-millisecond Accuracy

A simple method for synchronization of video streams with a precision better than one millisecond is proposed. The method is applicable to any number of rolling shutter cameras and when a few photographic flashes or other abrupt lighting changes are present in the video. The approach exploits the rolling shutter sensor property that every sensor row starts its exposure with a small delay after the onset of the previous row. The cameras may have different frame rates and resolutions, and need not have overlapping fields of view. The method was validated on five minutes of four streams from an ice hockey match. The found transformation maps events visible in all cameras to a reference time with a standard deviation of the temporal error in the range of 0.3 to 0.5 milliseconds. The quality of the synchronization is demonstrated on temporally and spatially overlapping images of a fast moving puck observed in two cameras.

[1]  Hans-Peter Seidel,et al.  Time-resolved 3d capture of non-stationary gas flows , 2008, SIGGRAPH 2008.

[2]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[3]  Gideon P. Stein,et al.  Tracking from multiple view points: Self-calibration of space and time , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Ian D. Reid,et al.  Synchronizing Image Sequences of Non-Rigid Objects , 2003, BMVC.

[5]  Ian D. Reid,et al.  Video synchronization from human motion using rank constraints , 2009, Comput. Vis. Image Underst..

[6]  Brian E. Granger,et al.  IPython: A System for Interactive Scientific Computing , 2007, Computing in Science & Engineering.

[7]  J. Fowles,et al.  RELATIONSHIPS TO SKATING PERFORMANCE IN COMPETITIVE HOCKEY PLAYERS , 2007, Journal of strength and conditioning research.

[8]  Kiriakos N. Kutulakos,et al.  Linear Sequence-to-Sequence Alignment , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Denis Simakov,et al.  Feature-Based Sequence-to-Sequence Matching , 2006, International Journal of Computer Vision.

[10]  Matis Hudon,et al.  High Speed Sequential Illumination With Electronic Rolling Shutter Cameras , 2015, CVPR 2015.

[11]  Darren J. Stefanyshyn,et al.  The influence of shaft stiffness on potential energy and puck speed during wrist and slap shots in ice hockey , 2006 .

[12]  Hans-Peter Seidel,et al.  Time-resolved 3d capture of non-stationary gas flows , 2008, SIGGRAPH Asia '08.

[13]  Derek Bradley,et al.  Synchronization and rolling shutter compensation for consumer video camera arrays , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[14]  Xin Li,et al.  Subframe Video Synchronization via 3D Phase Correlation , 2006, 2006 International Conference on Image Processing.

[15]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[16]  Hans Weda,et al.  Synchronization of multiple video recordings based on still camera flashes , 2006, MM '06.

[17]  Cheng Lei,et al.  Tri-focal tensor-based multiple video synchronization with subframe optimization , 2006, IEEE Transactions on Image Processing.

[18]  M. Irani,et al.  Spatio-Temporal Alignment of Sequences , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Marc Levoy,et al.  High-speed videography using a dense camera array , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..