Vehicle Re-identification with Learned Representation and Spatial Verification and Abnormality Detection with Multi-Adaptive Vehicle Detectors for Traffic Video Analysis

Traffic flow analysis is essential for intelligent transportation systems. In this paper, we propose methods for two challenging problems in traffic flow analysis: vehicle re-identification and abnormal event detection. For the first problem, we propose to combine learned high-level features for vehicle instance representation with hand-crafted local features for spatial verification. For the second problem, we propose to use multiple adaptive vehicle detectors for anomaly proposal and use heuristics properties extracted from anomaly proposals to determine anomaly events. Experiments on the datasets of traffic flow analysis from AI City Challenge 2019 show that our methods achieve mAP of 0.4008 for vehicle re-identification in Track 2, and can detect abnormal events with very high accuracy (F1 = 0.9429) in Track 3.

[1]  Yu Cheng,et al.  Dual-Mode Vehicle Motion Pattern Learning for High Performance Road Traffic Anomaly Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Shenghua Gao,et al.  Future Frame Prediction for Anomaly Detection - A New Baseline , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Wu Liu,et al.  Large-scale vehicle re-identification in urban surveillance videos , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[4]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[5]  Ling Shao,et al.  Viewpoint-Aware Attentive Multi-view Inference for Vehicle Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[7]  Jin-Hee Lee,et al.  ResNet-Based Vehicle Classification and Localization in Traffic Surveillance Systems , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Moritz Kampelmühler,et al.  Camera-based vehicle velocity estimation from monocular video , 2018, ArXiv.

[9]  Yanyun Zhao,et al.  Unsupervised Anomaly Detection for Traffic Surveillance Based on Background Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Tao Wang,et al.  Efficient Scene Layout Aware Object Detection for Traffic Surveillance , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Liang Zheng,et al.  Re-ranking Person Re-identification with k-Reciprocal Encoding , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Duy-Dinh Le,et al.  A Combination of Spatial Pyramid and Inverted Index for Large-Scale Image Retrieval , 2015, Int. J. Multim. Data Eng. Manag..

[13]  Xiaogang Wang,et al.  Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-identification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Bernt Schiele,et al.  Parameter-Free Spatial Attention Network for Person Re-Identification , 2018, ArXiv.

[15]  Maribel Yasmina Santos,et al.  Anomaly Detection in Roads with a Data Mining Approach , 2017, CENTERIS/ProjMAN/HCist.

[16]  Alessandro Perina,et al.  Analyzing Tracklets for the Detection of Abnormal Crowd Behavior , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[17]  Pavel Zemcík,et al.  Holistic recognition of low quality license plates by CNN using track annotated data , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[18]  Nicu Sebe,et al.  Abnormal event detection in videos using generative adversarial nets , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[19]  Junsong Yuan,et al.  Sparse reconstruction cost for abnormal event detection , 2011, CVPR 2011.

[20]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[21]  Qinghua Huang,et al.  Traffic anomaly detection based on image descriptor in videos , 2015, Multimedia Tools and Applications.

[22]  Xiaogang Wang,et al.  Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-Temporal Path Proposals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Tao Mei,et al.  PROVID: Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance , 2018, IEEE Transactions on Multimedia.

[26]  Andrew Zisserman,et al.  Three things everyone should know to improve object retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[28]  Peter Kontschieder,et al.  The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Minh N. Do,et al.  Traffic Flow Analysis with Multiple Adaptive Vehicle Detectors and Velocity Estimation with Landmark-Based Scanlines , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Mohsen Guizani,et al.  Vehicle Tracking Using Surveillance With Multimodal Data Fusion , 2018, IEEE Transactions on Intelligent Transportation Systems.

[31]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jun-Wei Hsieh,et al.  Automatic traffic surveillance system for vehicle tracking and classification , 2006, IEEE Transactions on Intelligent Transportation Systems.

[33]  Nicu Sebe,et al.  Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection , 2016, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[34]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[35]  Huchuan Lu,et al.  Combining motion and appearance cues for anomaly detection , 2016, Pattern Recognit..

[36]  Song-Chun Zhu,et al.  Recognizing Car Fluents from Video , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Maria Riveiro,et al.  Anomaly Detection for Road Traffic: A Visual Analytics Framework , 2017, IEEE Transactions on Intelligent Transportation Systems.

[38]  Hala H. Zayed,et al.  Day/Night Detector for Vehicle Tracking in Traffic Monitoring Systems , 2016 .

[39]  Mubarak Shah,et al.  Real-World Anomaly Detection in Surveillance Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Xiaodong Wang,et al.  Multi-Target Position and Velocity Estimation Using OFDM Communication Signals , 2019, IEEE Transactions on Communications.

[41]  Kil-Taek Lim,et al.  Vehicle Type Classification Using Bagging and Convolutional Neural Network on Multi View Surveillance Image , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[42]  Duc Anh Duong,et al.  Video instance search via spatial fusion of visual words and object proposals , 2019, International Journal of Multimedia Information Retrieval.

[43]  David Beymer,et al.  A real-time computer vision system for vehicle tracking and traffic surveillance , 1998 .

[44]  Yu Liu,et al.  Fast anomaly detection in traffic surveillance video based on robust sparse optical flow , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[45]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[46]  Tiejun Huang,et al.  Deep Relative Distance Learning: Tell the Difference between Similar Vehicles , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).