A nonparametric Riemannian framework on tensor field with application to foreground segmentation

Background modelling on tensor field has recently been proposed for foreground detection tasks. Taking into account the Riemannian structure of the tensor manifold, recent research has focused on developing parametric methods on the tensor domain e.g. gaussians mixtures (GMM) [7]. However, in some scenarios, simple parametric models do not accurately explain the physical processes. Kernel density estimators (KDE) have been successful to model, on Euclidean sample spaces the nonparametric nature of complex, time varying, and non-static backgrounds [8]. Founded on the mathematically rigorous KDE paradigm on general Riemannian manifolds [15], we define a KDE specifically to operate on the tensor manifold. We present a mathematically-sound framework for nonparametric modeling on tensor field to foreground segmentation. We endow the tensor manifold with two well-founded Riemannian metrics, i.e. Affine-Invariant and Log-Euclidean. Theoretical aspects are defined and the metrics are compared experimentally. Theoretic analysis and experimental results demonstrate the promise/effectiveness of the framework.

[1]  M. M. G. Ricci,et al.  Méthodes de calcul différentiel absolu et leurs applications , 1900 .

[2]  Memoria di T. Levi-Civita Nozione di parallelismo in una varietà qualunque e conseguente specificazione geometrica della curvatura riemanniana , 1916 .

[3]  I. Holopainen Riemannian Geometry , 1927, Nature.

[4]  W. Boothby An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[5]  T. Willmore MANIFOLDS ALL OF WHOSE GEODESICS ARE CLOSED , 1979 .

[6]  C. Atkinson Rao's distance measure , 1981 .

[7]  L. Skovgaard A Riemannian geometry of the multivariate normal model , 1984 .

[8]  Silvano Di Zenzo,et al.  A note on the gradient of a multi-image , 1986, Comput. Vis. Graph. Image Process..

[9]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[10]  J. Bigun Recognition of local symmetries in gray value images by harmonic functions , 1988 .

[11]  Johan Wiklund,et al.  Multidimensional Orientation Estimation with Applications to Texture Analysis and Optical Flow , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[13]  I. Chavel Riemannian Geometry: Subject Index , 2006 .

[14]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[15]  A. Ryan Russell , 1996 .

[16]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Stuart J. Russell,et al.  Image Segmentation in Video Sequences: A Probabilistic Approach , 1997, UAI.

[18]  E. García‐Río,et al.  Semi-Riemannian Manifolds , 1999 .

[19]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[20]  Azriel Rosenfeld,et al.  Detection and location of people in video images using adaptive fusion of color and edge information , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[23]  Mubarak Shah,et al.  A hierarchical approach to robust background subtraction using color and gradient information , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[24]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[25]  John M. Lee Introduction to Smooth Manifolds , 2002 .

[26]  Kai-Kuang Ma,et al.  Accurate optical flow estimation using adaptive scale-space and 3D structure tensor , 2002, Proceedings. International Conference on Image Processing.

[27]  Mohan M. Trivedi,et al.  Detecting Moving Shadows: Algorithms and Evaluation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Nikos Paragios,et al.  Background modeling and subtraction of dynamic scenes , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Kazuhiko Sumi,et al.  Background subtraction based on cooccurrence of image variations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[31]  Rachid Deriche,et al.  Tensor Processing for Texture and Colour Segmentation , 2005, SCIA.

[32]  Bruno Pelletier Kernel density estimation on Riemannian manifolds , 2005 .

[33]  Fatih Murat Porikli,et al.  A Bayesian Approach to Background Modeling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[34]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[35]  Robert Pless,et al.  Spatio-temporal Background Models for Outdoor Surveillance , 2005, EURASIP J. Adv. Signal Process..

[36]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[38]  Xavier Pennec,et al.  Statistical Computing on Manifolds for Computational Anatomy , 2006 .

[39]  Thrasyvoulos N. Pappas,et al.  Spatiotemporal algorithm for joint video segmentation and foreground detection , 2006, 2006 14th European Signal Processing Conference.

[40]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Rachid Deriche,et al.  DTI segmentation by statistical surface evolution , 2006, IEEE Transactions on Medical Imaging.

[42]  Rachid Deriche,et al.  Statistics on the Manifold of Multivariate Normal Distributions: Theory and Application to Diffusion Tensor MRI Processing , 2006, Journal of Mathematical Imaging and Vision.

[43]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[44]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[45]  Xavier Pennec,et al.  Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geometric Measurements , 2006, Journal of Mathematical Imaging and Vision.

[46]  Edwin R. Hancock,et al.  A Riemannian approach to graph embedding , 2007, Pattern Recognit..

[47]  Patrick Pérez,et al.  Detection and segmentation of moving objects in highly dynamic scenes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Yogesh Rathi,et al.  A Graph Cut Approach to Image Segmentation in Tensor Space , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Nicolas Martel-Brisson,et al.  Learning and Removing Cast Shadows through a Multidistribution Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Suyash P. Awate,et al.  Fuzzy Nonparametric DTI Segmentation for Robust Cingulum-Tract Extraction , 2007, MICCAI.

[51]  Jean-Marc Odobez,et al.  Multi-Layer Background Subtraction Based on Color and Texture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Yi-Ping Hung,et al.  Efficient hierarchical method for background subtraction , 2007, Pattern Recognit..

[53]  N. Ayache,et al.  Clinical DT-MRI Estimation, Smoothing, and Fiber Tracking With Log-Euclidean Metrics , 2007 .

[54]  Shengping Zhang,et al.  Dynamic background modeling and subtraction using spatio-temporal local binary patterns , 2008, 2008 15th IEEE International Conference on Image Processing.

[55]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Deborah Estrin,et al.  Background Subtraction on Distributions , 2008, ECCV.

[57]  Vassilios Morellas,et al.  Robust Foreground Detection In Video Using Pixel Layers , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Thierry Bouwmans,et al.  Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey , 2008 .

[59]  Shireen Elhabian,et al.  Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art , 2008 .

[60]  Nuno Vasconcelos,et al.  Background subtraction in highly dynamic scenes , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Pascal Fua,et al.  Making Background Subtraction Robust to Sudden Illumination Changes , 2008, ECCV.

[62]  Minglun Gong,et al.  Realtime background subtraction from dynamic scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[63]  Nikos Paragios,et al.  Scene modeling and change detection in dynamic scenes: A subspace approach , 2009, Comput. Vis. Image Underst..

[64]  Takeo Kanade,et al.  Background Subtraction for Freely Moving Cameras , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[65]  Jian Yang,et al.  A model for dynamic object segmentation with kernel density estimation based on gradient features , 2009, Image Vis. Comput..

[66]  Shengping Zhang,et al.  Dynamic Background Subtraction Based on Local Dependency Histogram , 2008, Int. J. Pattern Recognit. Artif. Intell..

[67]  Patrick Pérez,et al.  Detection and segmentation of moving objects in complex scenes , 2009, Comput. Vis. Image Underst..

[68]  Thierry Bouwmans,et al.  Subspace Learning for Background Modeling: A Survey , 2009 .

[69]  Hongxun Yao,et al.  Local Histogram of Figure/Ground Segmentations for Dynamic Background Subtraction , 2010, EURASIP J. Adv. Signal Process..

[70]  Matti Pietikäinen,et al.  Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[71]  Nuno Vasconcelos,et al.  Spatiotemporal Saliency in Dynamic Scenes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Bertrand Vachon,et al.  Statistical Background Modeling for Foreground Detection: A Survey , 2010 .

[73]  Rui Caseiro,et al.  Foreground Segmentation via Background Modeling on Riemannian Manifolds , 2010, 2010 20th International Conference on Pattern Recognition.

[74]  Nuno Vasconcelos,et al.  Generalized Stauffer–Grimson background subtraction for dynamic scenes , 2011, Machine Vision and Applications.

[75]  Rui Caseiro,et al.  Background Modelling on Tensor Field for Foreground Segmentation , 2010, BMVC.

[76]  Fan Zhang,et al.  New Riemannian techniques for directional and tensorial image data , 2010, Pattern Recognit..

[77]  Minglun Gong Foreground segmentation of live videos using locally competing 1SVMs , 2011, CVPR 2011.

[78]  Wenbing Tao,et al.  Texture segmentation using independent-scale component-wise Riemannian-covariance Gaussian mixture model in KL measure based multi-scale nonlinear structure tensor space , 2011, Pattern Recognit..

[79]  Rui Caseiro,et al.  A nonparametric Riemannian framework on tensor field with application to foreground segmentation , 2012, Pattern Recognit..