Sub-millisecond Video Synchronization of Multiple Android Smartphones

This paper addresses the problem of building an affordable easy-to-setup synchronized multi-view camera system, which is in demand for many Computer Vision and Robotics applications in high-dynamic environments. In our work, we propose a solution for this problem — a publicly-available Android application for synchronized video recording on multiple smartphones with sub-millisecond accuracy. We present a generalized mathematical model of timestamping for Android smartphones and prove its applicability on 47 different physical devices. Also, we estimate the time drift parameter for those smartphones, which is less than 1.2 ms per minute for most of the considered devices, that makes smartphones’ camera system a worthy analog for professional multi-view systems. Finally, we demonstrate Android-app performance on the camera system built from Android smartphones quantitatively on setup with lights and qualitatively — on panorama stitching task.

[1]  Dimitris Bertsimas,et al.  Online Mixed-Integer Optimization in Milliseconds , 2019, INFORMS J. Comput..

[2]  Ashok Veeraraghavan,et al.  SocialSync: Sub-Frame Synchronization in a Smartphone Camera Network , 2014, ECCV Workshops.

[3]  Brief Industry Paper: The Matter of Time — A General and Efficient System for Precise Sensor Synchronization in Robotic Computing , 2021, 2021 IEEE 27th Real-Time and Embedded Technology and Applications Symposium (RTAS).

[4]  Stephen P. Boyd,et al.  Embedded Mixed-Integer Quadratic optimization Using the OSQP Solver , 2018, 2018 European Control Conference (ECC).

[5]  Hujun Bao,et al.  Fast and Robust Multi-Person 3D Pose Estimation From Multiple Views , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Hirokazu Kameoka,et al.  Self-localization and channel synchronization of smartphone arrays using sound emissions , 2016, 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[7]  Marsel Faizullin,et al.  Twist-n-Sync: Software Clock Synchronization with Microseconds Accuracy Using MEMS-Gyroscopes , 2020, Sensors.

[8]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[9]  Rahul Garg,et al.  Wireless Software Synchronization of Multiple Distributed Cameras , 2018, 2019 IEEE International Conference on Computational Photography (ICCP).

[10]  Wonpil Yu,et al.  Efficient RBS in Sensor Networks , 2006, Third International Conference on Information Technology: New Generations (ITNG'06).

[11]  Yinda Zhang,et al.  Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels , 2020, ECCV.

[12]  Vladlen Koltun,et al.  OpenBot: Turning Smartphones into Robots , 2020, ArXiv.

[13]  Mani Srivastava,et al.  Exploiting Smartphone Peripherals for Precise Time Synchronization , 2019, 2019 IEEE International Symposium on Precision Clock Synchronization for Measurement, Control, and Communication (ISPCS).

[14]  Wenjun Zeng,et al.  Cross View Fusion for 3D Human Pose Estimation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Jiri Matas,et al.  Rolling Shutter Camera Synchronization with Sub-millisecond Accuracy , 2017, VISIGRAPP.

[17]  Mikhail Salnikov,et al.  Deep Learning Framework For Mobile Microscopy , 2020, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[18]  David L. Mills,et al.  Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI , 1996, RFC.

[19]  M. Bortolon,et al.  Multi-view data capture for dynamic object reconstruction using handheld augmented reality mobiles , 2021, J. Real Time Image Process..