Pedestrian tracking in thermal videos using TFM (tri-feature matrix)

Nowadays, thermal cameras play a major role because of its temperature-based photography in many applications such as video surveillance, monitoring electronics/electrical machines, forest monitoring, monitoring babies/adult patients, and suspicious object detection. Tracking pedestrians in thermal video is a major task for such applications. Thermal cameras usually create images based on temperature emitted by the object only and not on the lighting conditions and outdoor environment conditions. But still thermal images have constraints like no texture or colour information, more number of dead pixels, low resolution, and noticeable visual colour patterns in case of any temperature variations. So the challenge in tracking pedestrians in thermal videos is tracking objects/pedestrians throughout the video without an identity switch by overcoming these constraints which may mislead the tracking process. To overcome these constraints, the proposed system uses tri feature matrix (TFM) as an object descriptor which is used to uniquely identify and represent objects in thermal images. TFM is represented in more compact way as a triple matrix. It is a simple and accurate descriptor suitable for tracking objects in thermal video sequences without an identity switch. The proposed Pedestrian tracking system uses most of the advantages of thermal cameras by overcoming challenges in thermal videos effectively based on a novel descriptor TFM. The proposed system is evaluated with various data sets, and the results are analysed using true positive, true negative, false negative, false positive, accuracy, precision, recall, F-score, global identity mismatch (GMME) and track matching error (TME). The performance metrics such as accuracy, precision, recall, F-score, GMME and TME are computed as 99%, 100%, 99%, 99%, 2.3%, and 2.1%, respectively. From the observation, it is found that the performance of proposed TFM-based system is significantly improved. The experimental result shows that the proposed system achieved more accurate tracking compared to the conventional methods.

[1]  Wonjun Kim,et al.  Thermal Sensor-Based Multiple Object Tracking for Intelligent Livestock Breeding , 2017, IEEE Access.

[2]  Rustam Stolkin,et al.  Particle Filter Tracking of Camouflaged Targets by Adaptive Fusion of Thermal and Visible Spectra Camera Data , 2014, IEEE Sensors Journal.

[3]  Xinkai Wu,et al.  Pedestrian Detection and Tracking from Low-Resolution Unmanned Aerial Vehicle Thermal Imagery , 2016, Sensors.

[4]  D. Jude Hemanth,et al.  A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition , 2018, Comput. Intell..

[5]  Michael Felsberg,et al.  Channel Coded Distribution Field Tracking for Thermal Infrared Imagery , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Shehzad Khalid,et al.  Motion Trajectory Learning in the DFT-Coefficient Feature Space , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[7]  Lawrence Henesey,et al.  A 3D Convolutional Neural Network for Bacterial Image Classification , 2020 .

[8]  ByoungChul Ko,et al.  Pedestrian Tracking Using Online Boosted Random Ferns Learning in Far-Infrared Imagery for Safe Driving at Night , 2017, IEEE Transactions on Intelligent Transportation Systems.

[9]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  A. Enis Çetin,et al.  Co-difference based object tracking algorithm for infrared videos , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[11]  Kang Ryoung Park,et al.  Robust Pedestrian Detection by Combining Visible and Thermal Infrared Cameras , 2015, Sensors.

[12]  Mohd Asyraf Zulkifley,et al.  Two Streams Multiple-Model Object Tracker for Thermal Infrared Video , 2019, IEEE Access.

[13]  Ravi Kumar Jatoth,et al.  Performance evaluation of Alpha-Beta and Kalman filter for object tracking , 2014, 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies.

[14]  Brian C. Lovell,et al.  Detecting kangaroos in the wild: the first step towards automated animal surveillance , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Thomas B. Moeslund,et al.  Thermal cameras and applications: a survey , 2013, Machine Vision and Applications.

[16]  Qi Hao,et al.  Multiple Human Tracking and Identification With Wireless Distributed Pyroelectric Sensor Systems , 2009, IEEE Systems Journal.

[17]  Abhishikt Kadam.  Vehicle Automation and Car-Following Models for Accident Avoidance , 2020 .

[18]  Thaier Hayajneh,et al.  Lightweight Block Ciphers for IoT: Energy Optimization and Survivability Techniques , 2018, IEEE Access.

[19]  A. Aydin Alatan,et al.  Evaluation of Feature Channels for Correlation-Filter-Based Visual Object Tracking in Infrared Spectrum , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  K. Martin Sagayam,et al.  Image Fusion based on Sparse Sampling Method and Hybrid Discrete Cosine Transformation , 2019 .

[21]  Jin Tang,et al.  Grayscale-Thermal Object Tracking via Multitask Laplacian Sparse Representation , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Fahad Shahbaz Khan,et al.  Synthetic Data Generation for End-to-End Thermal Infrared Tracking , 2018, IEEE Transactions on Image Processing.

[23]  Huseyin Seckin Demir,et al.  Part-Based Co-Difference Object Tracking Algorithm for Infrared Videos , 2018, ICIP.

[24]  X. Mercilin Raajini,et al.  A statistical approach for high order epistasis interaction detection for prediction of diabetic macular edema , 2020 .

[25]  Niki Trigoni,et al.  Multiple-Model Fully Convolutional Neural Networks for Single Object Tracking on Thermal Infrared Video , 2018, IEEE Access.