Toward Planet-Wide Traffic Camera Calibration

Despite the widespread deployment of outdoor cameras, their potential for automated analysis remains largely untapped due, in part, to calibration challenges. The absence of precise camera calibration data, including intrinsic and extrinsic parameters, hinders accurate real-world distance measurements from captured videos. To address this, we present a scalable framework that utilizes street-level imagery to reconstruct a metric 3D model, facilitating precise calibration of in-the-wild traffic cameras. Notably, our framework achieves 3D scene reconstruction and accurate localization of over 100 global traffic cameras and is scalable to any camera with sufficient street-level imagery. For evaluation, we introduce a dataset of 20 fully calibrated traffic cameras, demonstrating our method's significant enhancements over existing automatic calibration techniques. Furthermore, we highlight our approach's utility in traffic analysis by extracting insights via 3D vehicle reconstruction and speed measurement, thereby opening up the potential of using outdoor cameras for automated analysis.

[1]  M. Pollefeys,et al.  LightGlue: Local Feature Matching at Light Speed , 2023, 2023 IEEE/CVF International Conference on Computer Vision (ICCV).

[2]  Y. Demiris,et al.  Monocular Visual Traffic Surveillance: A Review , 2022, IEEE Transactions on Intelligent Transportation Systems.

[3]  A. Schwing,et al.  Masked-attention Mask Transformer for Universal Image Segmentation , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jérôme Revaud,et al.  Robust Automatic Monocular Vehicle Speed Estimation for Traffic Surveillance , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Viktor Kocur,et al.  Traffic Camera Calibration via Vehicle Vanishing Point Detection , 2021, ICANN.

[6]  A. Herout,et al.  PlaneCalib: Automatic Camera Calibration by Multiple Observations of Rigid Objects on Plane , 2020, 2020 Digital Image Computing: Techniques and Applications (DICTA).

[7]  Chi-Keung Tang,et al.  GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision , 2020, ECCV.

[8]  Tomasz Malisiewicz,et al.  SuperGlue: Learning Feature Matching With Graph Neural Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zhigang Deng,et al.  Joint Prediction for Kinematic Trajectories in Vehicle-Pedestrian-Mixed Scenes , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Adam Herout,et al.  OptInOpt: Dual Optimization for Automatic Camera Calibration by Multi-Target Observations , 2019, 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[11]  Yubin Kuang,et al.  Deep Single Image Camera Calibration With Radial Distortion , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  N. Dinesh Reddy,et al.  Occlusion-Net: 2D/3D Occluded Keypoint Localization Using Graph Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Jean Charles Bazin,et al.  DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras , 2018, CVMP '18.

[14]  Jean-Baptiste Lamare,et al.  CADP: A Novel Dataset for CCTV Traffic Camera based Accident Analysis , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[15]  Julian Nubert,et al.  Traffic Density Estimation using a Convolutional Neural Network , 2018, ArXiv.

[16]  Yiannis Kompatsiaris,et al.  Speed Estimation and Abnormality Detection from Surveillance Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Tomasz Malisiewicz,et al.  SuperPoint: Self-Supervised Interest Point Detection and Description , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Prasun Sinha,et al.  Autocalib: automatic traffic camera calibration at scale , 2017, BuildSys@SenSys.

[19]  Yinhai Wang,et al.  Video Analytics towards Vision Zero , 2017 .

[20]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[21]  Adam Herout,et al.  Comprehensive Data Set for Automatic Single Camera Visual Speed Measurement , 2017, IEEE Transactions on Intelligent Transportation Systems.

[22]  Adam Herout,et al.  Traffic surveillance camera calibration by 3D model bounding box alignment for accurate vehicle speed measurement , 2017, Comput. Vis. Image Underst..

[23]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[25]  Andrew Zisserman,et al.  All About VLAD , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  H. Bischof,et al.  From structure-from-motion point clouds to fast location recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Fei-Yue Wang,et al.  Research on lane-marking line based camera calibration , 2007, 2007 IEEE International Conference on Vehicular Electronics and Safety.

[28]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[29]  Aljoscha Smolic,et al.  3-D reconstruction of a dynamic environment with a fully calibrated background for traffic scenes , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  A. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[33]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[34]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[35]  Sabbani imad,et al.  Deep convolutional neural network architecture for urban traffic flow estimation , 2018 .

[36]  Bernhard P. Wrobel Multiple View Geometry in Computer Vision , 2001, Künstliche Intell..

[37]  Noname manuscript No. (will be inserted by the editor) EPnP: An Accurate O(n) Solution to the PnP Problem , 2022 .