Deep-learning based single object tracker for night surveillance

Tracking an object in night surveillance video is a challenging task as the quality of the captured image is normally poor with low brightness and contrast. The task becomes harder for a small object as fewer features are apparent. Traditional approach is based on improving the image quality before tracking is performed. In this paper, a single object tracking algorithm based on deep-learning approach is proposed to exploit its outstanding capability of modelling object’s appearance even during night. The algorithm uses pre-trained convolutional neural networks coupled with fully connected layers, which are trained online during the tracking so that it is able to cater for appearance changes as the object moves around. Various learning hyperparameters for the optimization function, learning rate and ratio of training samples are tested to find optimal setup for tracking in night scenarios. Fourteen night surveillance videos are collected for validation purpose, which are captured from three viewing angles. The results show that the best accuracy is obtained by using Adam optimizer with learning rate of 0.00075 and sampling ratio of 2:1 for positive and negative training data. This algorithm is suitable to be implemented in higher level surveillance applications such as abnormal behavioral recognition.

[1]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[2]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[3]  Mohd Asyraf Zulkifley,et al.  Squat Angle Assessment Through Tracking Body Movements , 2019, IEEE Access.

[4]  Jiaolong Xu,et al.  Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison , 2016, Sensors.

[5]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Kang Ryoung Park,et al.  Human Detection Based on the Generation of a Background Image by Using a Far-Infrared Light Camera , 2015, Sensors.

[7]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[8]  Tieniu Tan,et al.  Detecting and Tracking Distant Objects at Night Based on Human Visual System , 2006, ACCV.

[9]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[10]  Dushyant Kumar Singh,et al.  Review of human detection techniques in night vision , 2017, 2017 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET).

[11]  Mohd Asyraf Zulkifley,et al.  Pterygium-Net: a deep learning approach to pterygium detection and localization , 2019, Multimedia Tools and Applications.

[12]  W. Marsden.  I and J , 2012 .

[13]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Chil-Woo Lee,et al.  Object detection and tracking in night time video surveillance , 2013, 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[15]  Bohyung Han,et al.  Learning Multi-domain Convolutional Neural Networks for Visual Tracking , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Vincent Lepetit,et al.  Feature Harvesting for Tracking-by-Detection , 2006, ECCV.

[18]  Yoshua Bengio,et al.  Object Recognition with Gradient-Based Learning , 1999, Shape, Contour and Grouping in Computer Vision.

[19]  Tieniu Tan,et al.  A real-time object detecting and tracking system for outdoor night surveillance , 2008, Pattern Recognit..

[20]  Andrew Zisserman,et al.  Detect to Track and Track to Detect , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Tieniu Tan,et al.  Object detection and tracking for night surveillance based on salient contrast analysis , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[22]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[23]  Yingfeng Cai,et al.  Night-Time Vehicle Sensing in Far Infrared Image with Deep Learning , 2016, J. Sensors.

[24]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[25]  Kang Ryoung Park,et al.  Convolutional Neural Network-Based Human Detection in Nighttime Images Using Visible Light Camera Sensors , 2017, Sensors.

[26]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[27]  Bohyung Han,et al.  Modeling and Propagating CNNs in a Tree Structure for Visual Tracking , 2016, ArXiv.

[28]  Qing Tian,et al.  Vehicle Detection and Tracking at Night in Video Surveillance , 2013, Int. J. Online Biomed. Eng..

[29]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[30]  Silvio Savarese,et al.  Learning to Track: Online Multi-object Tracking by Decision Making , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Niki Trigoni,et al.  Multiple-Model Fully Convolutional Neural Networks for Single Object Tracking on Thermal Infrared Video , 2018, IEEE Access.

[32]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[33]  Bernt Schiele,et al.  Learning People Detectors for Tracking in Crowded Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Tae Young Han,et al.  Night vision pedestrian detection based on adaptive preprocessing using near infrared camera , 2016, 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia).

[36]  William Moran,et al.  Robust Observation Detection for Single Object Tracking: Deterministic and Probabilistic Patch-Based Approaches , 2012, Sensors.

[37]  Mohd Asyraf Zulkifley,et al.  Robust single object tracker based on kernelled patch of a fixed RGB camera , 2016 .

[38]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[39]  Mohd Asyraf Zulkifley,et al.  Two Streams Multiple-Model Object Tracker for Thermal Infrared Video , 2019, IEEE Access.