Mask Sparse Representation Based on Semantic Features for Thermal Infrared Target Tracking

Thermal infrared (TIR) target tracking is a challenging task as it entails learning an effective model to identify the target in the situation of poor target visibility and clutter background. The sparse representation, as a typical appearance modeling approach, has been successfully exploited in the TIR target tracking. However, the discriminative information of the target and its surrounding background is usually neglected in the sparse coding process. To address this issue, we propose a mask sparse representation (MaskSR) model ,which combines sparse coding together with high-level semantic features for TIR target tracking. We first obtain the pixel-wise labeling results of the target and its surrounding background in the last frame, and then use such results to train target-specific deep networks using a supervised manner. According to the output features of the deep networks, the high-level pixel-wise discriminative map of the target area is obtained. We introduce the binarized discriminative map as a mask template to the sparse representation and develop a novel algorithm to collaboratively represent the reliable target part and unreliable target part partitioned with the mask template, which explicitly indicates different discriminant capabilities by label 1 and 0. The proposed MaskSR model controls the superiority of the reliable target part in the reconstruction process via a weighted scheme. We solve this multi-parameter constrained problem by a customized alternating direction method of multipliers (ADMM) method. This model is applied to achieve TIR target tracking in the particle filter framework. To improve the sampling effectiveness and decrease the computation cost at the same time, a discriminative particle selection strategy based on kernelized correlation filter is proposed to replace the previous random sampling for searching useful candidates. Our proposed tracking method was tested on the VOT-TIR2016 benchmark. The experiment results show that the proposed method has a significant superiority compared with various state-of-the-art methods in TIR target tracking.

[1]  Robert Laganière,et al.  Scalable Kernel Correlation Filter with Sparse Feature Integration , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[2]  Huchuan Lu,et al.  Visual tracking via adaptive structural local sparse appearance model , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Guohua Gu,et al.  Infrared small target tracking based on sample constrained particle filtering and sparse representation , 2017 .

[4]  Hao Wu,et al.  Infrared Small Target Detection Based on Non-Convex Optimization with Lp-Norm Constraint , 2019, Remote. Sens..

[5]  Michael Felsberg,et al.  Discriminative Scale Space Tracking , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  A. Aydin Alatan,et al.  Evaluation of Feature Channels for Correlation-Filter-Based Visual Object Tracking in Infrared Spectrum , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Zhenyu He,et al.  Thermal infrared object tracking via Siamese convolutional neural networks , 2017, 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[8]  Zhenyu He,et al.  A multi-view model for visual tracking via correlation filters , 2016, Knowl. Based Syst..

[9]  Qingming Huang,et al.  Hedged Deep Tracking , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Matej Kristan,et al.  Deformable Parts Correlation Filters for Robust Visual Tracking , 2016, IEEE Transactions on Cybernetics.

[11]  Wei An,et al.  Joint detection and tracking of size-varying infrared targets based on block-wise sparse decomposition , 2016 .

[12]  Liang Lin,et al.  Learning Collaborative Sparse Representation for Grayscale-Thermal Tracking. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[13]  Ying Li,et al.  Real-time infrared target tracking based on ℓ1 minimization and compressive features. , 2014, Applied optics.

[14]  Thomas Mauthner,et al.  In defense of color-based model-free tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Mang Ye,et al.  Online Non-Negative Multi-Modality Feature Template Learning for RGB-Assisted Infrared Tracking , 2019, IEEE Access.

[16]  Yang Li,et al.  Reliable Patch Trackers: Robust visual tracking by exploiting reliable patches , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Zhen Shi,et al.  Hierarchical search strategy in particle filter framework to track infrared target , 2016, Neural Computing and Applications.

[18]  Niki Trigoni,et al.  Multiple-Model Fully Convolutional Neural Networks for Single Object Tracking on Thermal Infrared Video , 2018, IEEE Access.

[19]  Pong C. Yuen,et al.  Learning Modality-Consistency Feature Templates: A Robust RGB-Infrared Tracking System , 2019, IEEE Transactions on Industrial Electronics.

[20]  Zhenming Peng,et al.  Infrared Small Target Detection Based on Partial Sum of the Tensor Nuclear Norm , 2019, Remote. Sens..

[21]  Ming Tang,et al.  Multi-kernel Correlation Filter for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Min Li,et al.  Infrared Target Tracking Based on Robust Low-Rank Sparse Learning , 2016, IEEE Geoscience and Remote Sensing Letters.

[23]  Zhengzhou Li,et al.  Dim moving target tracking algorithm based on particle discriminative sparse representation , 2016 .

[24]  Gang Wang,et al.  Real-time part-based visual tracking via adaptive correlation filters , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Zhen Shi,et al.  A Parallel Search Strategy Based on Sparse Representation for Infrared Target Tracking , 2015, Algorithms.

[26]  Haibin Ling,et al.  Real time robust L1 tracker using accelerated proximal gradient approach , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Rui Caseiro,et al.  High-Speed Tracking with Kernelized Correlation Filters , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ming-Hsuan Yang,et al.  Hierarchical Convolutional Features for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Narendra Ahuja,et al.  Robust visual tracking via multi-task sparse learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Qifeng Yu,et al.  Dense structural learning for infrared object tracking at 200+ Frames per Second , 2017, Pattern Recognit. Lett..

[31]  Shengping Zhang,et al.  Modality-correlation-aware sparse representation for RGB-infrared object tracking , 2020, Pattern Recognit. Lett..

[32]  Jin Tang,et al.  Grayscale-Thermal Object Tracking via Multitask Laplacian Sparse Representation , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[33]  Aykut Erdem,et al.  Deformable part-based tracking by coupled global and local correlation filters , 2016, J. Vis. Commun. Image Represent..

[34]  Jin Tang,et al.  Weighted Sparse Representation Regularized Graph Learning for RGB-T Object Tracking , 2017, ACM Multimedia.

[35]  Zhetao Li,et al.  Visual Tracking With Weighted Adaptive Local Sparse Appearance Model via Spatio-Temporal Context Learning , 2018, IEEE Transactions on Image Processing.

[36]  Zhenyu He,et al.  Hierarchical spatial-aware Siamese network for thermal infrared object tracking , 2017, Knowl. Based Syst..

[37]  Xi Zhang,et al.  Layered Optical Flow Estimation Using a Deep Neural Network with a Soft Mask , 2018, IJCAI.

[38]  Zhenyu He,et al.  Deep convolutional neural networks for thermal infrared object tracking , 2017, Knowl. Based Syst..

[39]  Changsheng Xu,et al.  Robust Structural Sparse Tracking , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Qi Tian,et al.  Geometric Hypergraph Learning for Visual Tracking , 2016, IEEE Transactions on Cybernetics.

[41]  Somnath Sengupta,et al.  Rough-Set-Theoretic Fuzzy Cues-Based Object Tracking Under Improved Particle Filter Framework , 2016, IEEE Transactions on Fuzzy Systems.

[42]  Michael Felsberg,et al.  Channel Coded Distribution Field Tracking for Thermal Infrared Imagery , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[43]  Michael Felsberg,et al.  Learning Spatially Regularized Correlation Filters for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[44]  Fahad Shahbaz Khan,et al.  Synthetic Data Generation for End-to-End Thermal Infrared Tracking , 2018, IEEE Transactions on Image Processing.

[45]  Bingjian Wang,et al.  Infrared dim moving target tracking via sparsity-based discriminative classifier and convolutional network , 2017 .

[46]  Rongrong Ji,et al.  Robust infrared target tracking based on particle filter with embedded saliency detection , 2015, Inf. Sci..

[47]  Zhenming Peng,et al.  Infrared Small Target Detection via Non-Convex Rank Approximation Minimization Joint l2, 1 Norm , 2018, Remote. Sens..

[48]  Changsheng Xu,et al.  Learning Multi-Task Correlation Particle Filters for Visual Tracking , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Zhenyu He,et al.  The Thermal Infrared Visual Object Tracking VOT-TIR2016 Challenge Results , 2016, ECCV Workshops.

[50]  Xavier Maldague,et al.  Total Variation Regularization Term-Based Low-Rank and Sparse Matrix Representation Model for Infrared Moving Target Tracking , 2018, Remote. Sens..

[51]  Seong Tae Jhang,et al.  Infrared Target Tracking Using Multi-Feature Joint Sparse Representation , 2016, RACS.