Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking

Appearance modeling is very important for background modeling and object tracking. Subspace learning-based algorithms have been used to model the appearances of objects or scenes. Current vector subspace-based algorithms cannot effectively represent spatial correlations between pixel values. Current tensor subspace-based algorithms construct an offline representation of image ensembles, and current online tensor subspace learning algorithms cannot be applied to background modeling and object tracking. In this paper, we propose an online tensor subspace learning algorithm which models appearance changes by incrementally learning a tensor subspace representation through adaptively updating the sample mean and an eigenbasis for each unfolding matrix of the tensor. The proposed incremental tensor subspace learning algorithm is applied to foreground segmentation and object tracking for grayscale and color image sequences. The new background models capture the intrinsic spatiotemporal characteristics of scenes. The new tracking algorithm captures the appearance characteristics of an object during tracking and uses a particle filter to estimate the optimal object state. Experimental evaluations against state-of-the-art algorithms demonstrate the promise and effectiveness of the proposed incremental tensor subspace learning algorithm, and its applications to foreground segmentation and object tracking.

[1]  Deng Cai,et al.  Tensor Subspace Analysis , 2005, NIPS.

[2]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  David A. Forsyth,et al.  Tracking People by Learning Their Appearance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jimeng Sun,et al.  Beyond streams and graphs: dynamic tensor analysis , 2006, KDD '06.

[5]  Ming-Hsuan Yang,et al.  Incremental Learning for Robust Visual Tracking , 2008, International Journal of Computer Vision.

[6]  Shuicheng Yan,et al.  A Convengent Solution to Tensor Subspace Learning , 2007, IJCAI.

[7]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[8]  George Karypis,et al.  Introduction to special issue on bioinformatics , 2008, TKDD.

[9]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Wen Gao,et al.  Matrix-Structural Learning (MSL) of Cascaded Classifier from Enormous Training Set , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Ming Yang,et al.  Tracking Nonstationary Visual Appearances by Data-Driven Adaptation , 2009, IEEE Transactions on Image Processing.

[12]  Dong Xu,et al.  Discriminant analysis with tensor representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Narendra Ahuja,et al.  A Tensor Approximation Approach to Dimensionality Reduction , 2008, International Journal of Computer Vision.

[14]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[15]  Narendra Ahuja,et al.  Rank-R approximation of tensors using image-as-matrix representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[17]  Ying Wu,et al.  Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning , 2004, International Journal of Computer Vision.

[18]  Jieping Ye,et al.  GPCA: an efficient dimension reduction scheme for image compression and retrieval , 2004, KDD.

[19]  Yang Wang,et al.  A probabilistic approach for foreground and shadow segmentation in monocular image sequences , 2005, Pattern Recognit..

[20]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[21]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[22]  Jimeng Sun,et al.  Streaming Pattern Discovery in Multiple Time-Series , 2005, VLDB.

[23]  Horst Bischof,et al.  Semi-supervised On-Line Boosting for Robust Tracking , 2008, ECCV.

[24]  Luc Van Gool,et al.  An adaptive color-based particle filter , 2003, Image Vis. Comput..

[25]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[26]  Demetri Terzopoulos,et al.  Multilinear subspace analysis of image ensembles , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Stanley C. Eisenstat,et al.  Downdating the Singular Value Decomposition , 1995, SIAM J. Matrix Anal. Appl..

[28]  Larry S. Davis,et al.  Efficient mean-shift tracking via a new similarity measure , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  David J. Fleet,et al.  A framework for modeling appearance change in image sequences , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Jieping Ye,et al.  Generalized Low Rank Approximations of Matrices , 2004, Machine Learning.

[32]  Philip S. Yu,et al.  Incremental tensor analysis: Theory and applications , 2008, TKDD.

[33]  Yongmin Li,et al.  On incremental and robust subspace learning , 2004, Pattern Recognit..

[34]  Philip S. Yu,et al.  Window-based Tensor Analysis on High-dimensional and Multi-aspect Streams , 2006, Sixth International Conference on Data Mining (ICDM'06).

[35]  Max Lu,et al.  Robust and efficient foreground analysis for real-time video surveillance , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  David J. Kriegman,et al.  Online learning of probabilistic appearance manifolds for video-based recognition and tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[37]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[38]  Nuno Vasconcelos,et al.  Saliency-based discriminant tracking , 2009, CVPR.

[39]  D. Kriegman,et al.  Visual tracking using learned linear subspaces , 2004, CVPR 2004.

[40]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[41]  Hans-Peter Seidel,et al.  Drift-free tracking of rigid and articulated objects , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Kyoung Mu Lee,et al.  Tracking of a non-rigid object via patch-based dynamic appearance modeling and adaptive Basin Hopping Monte Carlo sampling , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Rainer Stiefelhagen,et al.  Dynamic Integration of Generalized Cues for Person Tracking , 2008, ECCV.

[44]  Mario Sznaier,et al.  Dynamic Appearance Modeling for Human Tracking , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[45]  Xiaoqin Zhang,et al.  Robust Visual Tracking Based on Incremental Tensor Subspace Learning , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[46]  Michael Lindenbaum,et al.  Sequential Karhunen-Loeve basis extraction and its application to images , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[47]  Soraia Raupp Musse,et al.  A Background Subtraction Model Adapted to Illumination Changes , 2006, 2006 International Conference on Image Processing.

[48]  Danijel Skocaj,et al.  Weighted and robust incremental method for subspace learning , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[49]  Gene H. Golub,et al.  Matrix computations , 1983 .

[50]  Roberto Cipolla,et al.  Robust Appearance-based Tracking using a sparse Bayesian classifier , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[51]  Yang Wang,et al.  A dynamic conditional random field model for foreground and shadow segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Rama Chellappa,et al.  Appearance modeling under geometric context , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[53]  Yaser Sheikh,et al.  Bayesian object detection in dynamic scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[54]  Junseok Kwon,et al.  Tracking of a non-rigid object via patch-based dynamic appearance modeling and adaptive Basin Hopping Monte Carlo sampling , 2009, CVPR.

[55]  Datong Chen,et al.  Robust Object Tracking Via Online Dynamic Spatial Bias Appearance Models , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[57]  Ying Wu,et al.  Differential Tracking based on Spatial-Appearance Model (SAM) , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[58]  Ying Chen,et al.  Markov Random Field Modeled Level Sets Method for Object Tracking with Moving Cameras , 2007, ACCV.

[59]  Bohyung Han,et al.  Visual Tracking by Continuous Density Propagation in Sequential Bayesian Filtering Framework , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Jieping Ye,et al.  Two-Dimensional Linear Discriminant Analysis , 2004, NIPS.

[61]  Takahiro Ishikawa,et al.  The template update problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Vassilios Morellas,et al.  Robust Foreground Detection In Video Using Pixel Layers , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).