Online Metric-Weighted Linear Representations for Robust Visual Tracking

In this paper, we propose a visual tracker based on a metric-weighted linear representation of appearance. In order to capture the interdependence of different feature dimensions, we develop two online distance metric learning methods using proximity comparison information and structured output learning. The learned metric is then incorporated into a linear representation of appearance. We show that online distance metric learning significantly improves the robustness of the tracker, especially on those sequences exhibiting drastic appearance changes. In order to bound growth in the number of training samples, we design a time-weighted reservoir sampling method. Moreover, we enable our tracker to automatically perform object identification during the process of object tracking, by introducing a collection of static template samples belonging to several object classes of interest. Object identification results for an entire video sequence are achieved by systematically combining the tracking information and visual recognition at each frame. Experimental results on challenging video sequences demonstrate the effectiveness of the method for both inter-frame tracking and object identification.

[1]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[2]  Shai Avidan,et al.  Support vector tracking , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[4]  Andreas E. Savakis,et al.  Online Distance Metric Learning for Object Tracking , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Ming Tang,et al.  Robust tracking via weakly supervised ranking SVM , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Xiaoqin Zhang,et al.  Visual tracking via incremental Log-Euclidean Riemannian subspace learning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[8]  Vibhav Vineet,et al.  Struck: Structured Output Tracking with Kernels , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[11]  Junseok Kwon,et al.  Robust visual tracking using autoregressive hidden Markov Model , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Zhongfei Zhang,et al.  Visual Tracking With Spatio-Temporal Dempster–Shafer Information Fusion , 2013, IEEE Transactions on Image Processing.

[13]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[14]  Samy Bengio,et al.  Large Scale Online Learning of Image Similarity Through Ranking , 2009, J. Mach. Learn. Res..

[15]  Lei Zhang,et al.  Real-Time Compressive Tracking , 2012, ECCV.

[16]  Gabriela Csurka,et al.  Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost , 2012, ECCV.

[17]  Haibin Ling,et al.  Robust Visual Tracking and Vehicle Classification via Sparse Representation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Narendra Ahuja,et al.  Robust Visual Tracking via Structured Multi-Task Sparse Learning , 2012, International Journal of Computer Vision.

[19]  Hanzi Wang,et al.  Incremental Learning of 3D-DCT Compact Representations for Robust Visual Tracking , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Gang Hua,et al.  Discriminative Tracking by Metric Learning , 2010, ECCV.

[21]  Huchuan Lu,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON IMAGE PROCESSING 1 Online Object Tracking with Sparse Prototypes , 2022 .

[22]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[23]  Stephen Tyree,et al.  Non-linear Metric Learning , 2012, NIPS.

[24]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[25]  Alston S. Householder,et al.  The Theory of Matrices in Numerical Analysis , 1964 .

[26]  Li Bai,et al.  Real-Time Probabilistic Covariance Tracking With Efficient Model Update , 2012, IEEE Transactions on Image Processing.

[27]  Timothy F. Cootes,et al.  Improving identification performance by integrating evidence from sequences , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  Haibin Ling,et al.  Real time robust L1 tracker using accelerated proximal gradient approach , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Zhibin Hong,et al.  Dual-Force Metric Learning for Robust Distracter-Resistant Tracker , 2012, ECCV.

[30]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Chunhua Shen,et al.  Real-time visual tracking using compressive sensing , 2011, CVPR 2011.

[32]  M. J. D. Powell,et al.  A Theorem on Rank One Modifications to a Matrix and Its Inverse , 1969, Comput. J..

[33]  Michael Kolonko,et al.  Sequential reservoir sampling with a nonuniform distribution , 2006, TOMS.

[34]  Ze-Nian Li,et al.  A large margin framework for single camera offline tracking with hybrid cues , 2012, Comput. Vis. Image Underst..

[35]  Huchuan Lu,et al.  Robust object tracking via sparsity-based collaborative model , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Ehud Rivlin,et al.  Robust Fragments-based Tracking using the Integral Histogram , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[37]  Junseok Kwon,et al.  Visual tracking decomposition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Ming-Hsuan Yang,et al.  Visual tracking with online Multiple Instance Learning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Horst Bischof,et al.  Real-Time Tracking via On-line Boosting , 2006, BMVC.

[40]  Gene H. Golub,et al.  Matrix computations , 1983 .

[41]  Vincent Lepetit,et al.  Are sparse representations really relevant for image classification? , 2011, CVPR 2011.

[42]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Ying Wu,et al.  Discriminative Spatial Attention for Robust Tracking , 2010, ECCV.

[44]  Nan Jiang,et al.  Adaptive and discriminative metric differential tracking , 2011, CVPR 2011.

[45]  Horst Bischof,et al.  PROST: Parallel robust online simple tracking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Rong Jin,et al.  Online AUC Maximization , 2011, ICML.

[47]  Ming-Hsuan Yang,et al.  Incremental Learning for Robust Visual Tracking , 2008, International Journal of Computer Vision.

[48]  Paul G. Spirakis,et al.  Weighted random sampling with a reservoir , 2006, Inf. Process. Lett..

[49]  Huchuan Lu,et al.  Visual tracking via adaptive structural local sparse appearance model , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Anton van den Hengel,et al.  Non-sparse linear representations for visual tracking with online reservoir metric learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.