Spatial-aware correlation filters with adaptive weight maps for visual tracking

Abstract Recently discriminative correlation filter (DCF) methods demonstrate excellent performance for visual tracking. These methods, however, often suffer from boundary effects, especially when the target search area is small. To mitigate boundary effects, a larger target search area is chosen, which, however, introduces many background pixels so that the tracking filter is contaminated by the introduced background pixels and the tracking performance is degraded. In order to alleviate the effects of the background pixels, this paper proposes a spatial-aware adaptive weight map, which wisely assigns large weights to pixels of the target and small weights to pixels in the background. Such desired weight map is adaptively generated by combining the target likelihood, which quantitatively measures whether a pixel belongs to the target or the background, and prior spatial weights of pixels. By integrating the spatial-aware adaptive weight map into the DCF framework, the obtained DCF tracking method can achieve more precise target location and suffer from less tracking drift than those conventional tracking methods. Moreover, we compute a confidence score from the response map and train the tracking filter only in the high-confidence situation. This high-confidence updating strategy can effectively avoid the tracking filter corruption problem when the target is severely occluded and the target search area is mainly filled with pixels of the background. The proposed tracking method is compared with some state-of-the-art methods through extensive experiments on four benchmark datasets, which confirm the performance superiority of the proposed one.

[1]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Michael Felsberg,et al.  Adaptive Color Attributes for Real-Time Visual Tracking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Simon Lucey,et al.  Multi-channel Correlation Filters , 2013, 2013 IEEE International Conference on Computer Vision.

[4]  Bernard Ghanem,et al.  Target Response Adaptation for Correlation Filter Tracking , 2016, ECCV.

[5]  Peng Liu,et al.  Spatial-temporal adaptive feature weighted correlation filter for visual tracking , 2018, Signal Process. Image Commun..

[6]  Simon Lucey,et al.  Learning Background-Aware Correlation Filters for Visual Tracking , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Luca Bertinetto,et al.  Staple: Complementary Learners for Real-Time Tracking , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Feng Li,et al.  Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Matej Kristan,et al.  A Graphical Model for Rapid Obstacle Image-Map Estimation from Unmanned Surface Vehicles , 2014, ACCV.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Bernard Ghanem,et al.  Context-Aware Correlation Filter Tracking , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Shiuh-Ku Weng,et al.  Video object tracking using adaptive Kalman filter , 2006, J. Vis. Commun. Image Represent..

[13]  Qingshan Liu,et al.  Parallel Attentive Correlation Tracking , 2019, IEEE Transactions on Image Processing.

[14]  Qi Wang,et al.  Multi-cue based tracking , 2014, Neurocomputing.

[15]  Yuhui Zheng,et al.  Robust visual tracking via self-similarity learning , 2017 .

[16]  Feiping Nie,et al.  Detecting Coherent Groups in Crowd Scenes by Multiview Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Michael Felsberg,et al.  Learning Spatially Regularized Correlation Filters for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Jiri Matas,et al.  Discriminative Correlation Filter with Channel and Spatial Reliability , 2017, CVPR.

[20]  Rui Caseiro,et al.  High-Speed Tracking with Kernelized Correlation Filters , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Luca Bertinetto,et al.  Fully-Convolutional Siamese Networks for Object Tracking , 2016, ECCV Workshops.

[22]  Junliang Xing,et al.  Learning Attentions: Residual Attentional Siamese Network for High Performance Online Visual Tracking , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Philip H. S. Torr,et al.  Struck: Structured output tracking with kernels , 2011, ICCV.

[24]  Qingshan Liu,et al.  Robust object tracking by online Fisher discrimination boosting feature selection , 2016, Comput. Vis. Image Underst..

[25]  Thomas Mauthner,et al.  In defense of color-based model-free tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[27]  Dit-Yan Yeung,et al.  Understanding and Diagnosing Visual Tracking Systems , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Yiannis Demiris,et al.  Attentional Correlation Filter Network for Adaptive Visual Tracking , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Qingshan Liu,et al.  Visual tracking using spatio-temporally nonlocally regularized correlation filter , 2018, Pattern Recognit..

[30]  Bruce A. Draper,et al.  Visual object tracking using adaptive correlation filters , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Michael Felsberg,et al.  Accurate Scale Estimation for Robust Visual Tracking , 2014, BMVC.

[32]  Jianke Zhu,et al.  A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration , 2014, ECCV Workshops.

[33]  Chong Luo,et al.  A Twofold Siamese Network for Real-Time Object Tracking , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Ming-Hsuan Yang,et al.  Long-term correlation tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Yi Wu,et al.  Online Object Tracking: A Benchmark , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Yong Liu,et al.  Large Margin Object Tracking with Circulant Feature Maps , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Wei Chen,et al.  Robust visual tracking via patch based kernel correlation filters with adaptive multiple feature ensemble , 2016, Neurocomputing.

[38]  Qingshan Liu,et al.  Complementary Tracking via Dual Color Clustering and Spatio-Temporal Regularized Correlation Learning , 2018, IEEE Access.

[39]  Zhenyu He,et al.  The Visual Object Tracking VOT2016 Challenge Results , 2016, ECCV Workshops.

[40]  Michael Felsberg,et al.  Convolutional Features for Correlation Filter Based Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[41]  Michael Felsberg,et al.  Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking , 2016, ECCV.

[42]  Jiri Matas,et al.  Robust scale-adaptive mean-shift for tracking , 2013, Pattern Recognition Letters.

[43]  Luca Bertinetto,et al.  End-to-End Representation Learning for Correlation Filter Based Tracking , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Rui Caseiro,et al.  Exploiting the Circulant Structure of Tracking-by-Detection with Kernels , 2012, ECCV.

[45]  Qingshan Liu,et al.  Visual Tracking via Boolean Map Representations , 2016, Pattern Recognit..

[46]  Qingshan Liu,et al.  Robust Visual Tracking via Convolutional Networks Without Training , 2016, IEEE Transactions on Image Processing.

[47]  Ming-Hsuan Yang,et al.  Hierarchical Convolutional Features for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[48]  Fan Yang,et al.  Efficient Correlation Tracking via Center-Biased Spatial Regularization , 2018, IEEE Transactions on Image Processing.

[49]  Ming-Hsuan Yang,et al.  Object Tracking Benchmark , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Xuelong Li,et al.  Hierarchical Feature Selection for Random Projection , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Gongjian Wen,et al.  Learning adaptively windowed correlation filters for robust tracking , 2018, J. Vis. Commun. Image Represent..

[52]  Cordelia Schmid,et al.  Learning Color Names for Real-World Applications , 2009, IEEE Transactions on Image Processing.

[53]  Simone Calderara,et al.  Visual Tracking: An Experimental Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Erik Blasch,et al.  Encoding color information for visual tracking: Algorithms and benchmark , 2015, IEEE Transactions on Image Processing.