Deep Learning-based Trajectory Estimation of Vehicles in Crowded and Crossroad Scenarios

The rapid developments in the field of Artificial Intelligence are bringing enhancements in the area of intelligent transport systems by overcoming the challenges of safety concerns. Traffic surveillance systems based on CCTV cameras can help us to achieve safe and sustainable transport systems. Trajectory estimation of vehicles is an important part of traffic surveillance systems and self-driving cars. The task is challenging due to the variations in illumination intensities, object sizes and real-time detection. We propose tracking by detection based trajectory estimation pipeline which consists of two stages: The first stage is the detection and localization of vehicles and the second stage is building associations in bounding boxes and track the associated bounding boxes. We analyze the performance of the Mask RCNN benchmark and YOLOv3 on the UA-DETRAC dataset and evaluate certain metrics like Intersection over Union, Precision-Recall curve, and Mean Average Precision. Experiments show that Mask RCNN Benchmark outperforms YOLOv3 in terms of accuracy. SORT tracker is applied on detected bounding boxes to estimate trajectories. The tracker is evaluated using mean absolute error. We demonstrate that the developed technique works successfully in crowded and crossroad scenarios.

[1]  Sandesh Shrestha,et al.  Vehicle Tracking Using Video Surveillance , 2019, Intelligent System and Computing.

[2]  Ilya Afanasyev,et al.  ROS-based Integration of Smart Space and a Mobile Robot as the Internet of Robotic Things , 2019, 2019 25th Conference of Open Innovations Association (FRUCT).

[3]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Vladimir Shepelev,et al.  Traffic flow estimation with data from a video surveillance camera , 2019, Journal of Big Data.

[5]  Yohan Dupuis,et al.  A Survey of Vision-Based Traffic Monitoring of Road Intersections , 2016, IEEE Transactions on Intelligent Transportation Systems.

[6]  Hong Cheng,et al.  Vehicle Detection and Tracking , 2011 .

[7]  Qian Zhang,et al.  An Improved YOLOv2 for Vehicle Detection , 2018, Sensors.

[8]  Anis Koubaa,et al.  Aerial Images Processing for Car Detection using Convolutional Neural Networks: Comparison between Faster R-CNN and YoloV3 , 2019, ArXiv.

[9]  Ming-Hsuan Yang,et al.  UA-DETRAC: A new benchmark and protocol for multi-object detection and tracking , 2015, Comput. Vis. Image Underst..

[10]  Jun-Cheng Chen,et al.  Towards Real-Time Systems for Vehicle Re-Identification, Multi-Camera Tracking, and Anomaly Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Volker Eiselein,et al.  High-Speed tracking-by-detection without using image information , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jaewan Choi,et al.  Determination of Vehicle Trajectory through Optimization of Vehicle Bounding Boxes using a Convolutional Neural Network , 2019, Sensors.

[14]  Saturnino Maldonado-Bascón,et al.  Boosting Multi-Vehicle Tracking with a Joint Object Detection and Viewpoint Estimation Sensor , 2019, Sensors.

[15]  Tobias Senst,et al.  Extending IOU Based Multi-Object Tracking by Visual Information , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[16]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[18]  David G. Dorrell,et al.  Online Parameter Identification of Ultracapacitor Models Using the Extended Kalman Filter , 2014 .

[19]  Edouard Ivanjko,et al.  Real Time Vehicle Trajectory Estimation on Multiple Lanes , 2014 .

[20]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[21]  Argel A. Bandala,et al.  Multi-view multi-object tracking in an intelligent transportation system: A literature review , 2017, 2017IEEE 9th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM).

[22]  Edouard Ivanjko,et al.  Measurement of Road Traffic Parameters Based on Multi-Vehicle Tracking , 2015, ArXiv.

[23]  Lourdes Agapito,et al.  Reconstructing PASCAL VOC , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Giuseppe Pirlo,et al.  Vehicular Traffic Congestion Classification by Visual Features and Deep Learning Approaches: A Comparison , 2019, Sensors.

[25]  Guizhong Liu,et al.  Multiple Object Tracking with Motion and Appearance Cues , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[26]  Jian Zhang,et al.  Deep learning for robust outdoor vehicle visual tracking , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[27]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[28]  Yi Wang,et al.  Vehicle Tracking Using Deep SORT with Low Confidence Track Filtering , 2019, 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[29]  Doo-Hyun Choi,et al.  Performance Enhancement of YOLOv3 by Adding Prediction Layers with Spatial Pyramid Pooling for Vehicle Detection , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[30]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[31]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .

[32]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[33]  Saeed Asadi Bagloee,et al.  Applications of Artificial Intelligence in Transport: An Overview , 2022 .

[34]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[35]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[36]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Jie Wang,et al.  Orientation- and Scale-Invariant Multi-Vehicle Detection and Tracking from Unmanned Aerial Videos , 2019, Remote. Sens..

[38]  Doo-Hyun Choi,et al.  Multi-Scale Detector for Accurate Vehicle Detection in Traffic Surveillance Data , 2019, IEEE Access.