Real-time online camera synchronization for volume carving on GPU

Volume carving is a well-known technique for reconstructing a 3D scene from a set of 2D images, using features detected in individual cameras, and camera parameters. Spatial calibration of the cameras is well understood, but the resulting carved volume is very sensitive to temporal offsets between the cameras. Automatic synchronization between the cameras is therefore desirable. In this paper, we present a highly efficient implementation of volume carving and synchronization on a heterogeneous system fitted with commodity GPUs using an improved version of the algorithm in [1]. An online, real-time synchronization system is described and evaluated on surveillance video of an indoor scene. Improvements to the state of the art CPU-based algorithms are described.

[1]  Takayuki Okatani,et al.  Video Synchronization Based on Co-occurrence of Appearance Changes in Video Sequences , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  Jesús M. Pérez,et al.  Real-time voxel-based visual hull reconstruction , 2012, Microprocess. Microsystems.

[3]  Hans-Peter Seidel,et al.  Markerless Motion Capture with unsynchronized moving cameras , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Jean-Yves Guillemaut,et al.  Through-the-Lens Multi-camera Synchronisation and Frame-Drop Detection for 3D Reconstruction , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[5]  Dariu Gavrila,et al.  Multi-view 3D Human Pose Estimation in Complex Environment , 2011, International Journal of Computer Vision.

[6]  Dariu Gavrila,et al.  Multi-person Localization and Track Assignment in Overlapping Camera Views , 2011, DAGM-Symposium.

[7]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[8]  Yael Moses,et al.  Video Synchronization Using Temporal Signals from Epipolar Lines , 2010, ECCV.

[9]  Prosenjit Bose,et al.  Temporal Synchronization of Video Sequences in Theory and in Practice , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[10]  Nassir Navab,et al.  Efficient visual hull computation for real-time 3D reconstruction using CUDA , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[11]  Peter Eisert,et al.  Parallel high resolution real-time Visual Hull on GPU , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).