Quaternion-based dynamic mode decomposition for background modeling in color videos

Scene Background Initialization (SBI) is one of the challenging problems in computer vision. Dynamic mode decomposition (DMD) is a recently proposed method to robustly decompose a video sequence into the background model and the corresponding foreground part. However, this method needs to convert the color image into the grayscale image for processing, which leads to the neglect of the coupling information between the three channels of the color image. In this study, we propose a quaternion-based DMD (Q-DMD), which extends the DMD by quaternion matrix analysis, so as to completely preserve the inherent color structure of the color image and the color video. We exploit the standard eigenvalues of the quaternion matrix to compute its spectral decomposition and calculate the corresponding Q-DMD modes and eigenvalues. The results on the publicly available benchmark datasets prove that our Q-DMD outperforms the exact DMDmethod, and experiment results also demonstrate that the performance of our approach is comparable to that of the state-of-the-art ones.

[1]  Stephen J. Sangwine,et al.  Hypercomplex Fourier Transforms of Color Images , 2001, IEEE Transactions on Image Processing.

[2]  Marc Van Droogenbroeck,et al.  Deep background subtraction with scene-specific convolutional neural networks , 2016, 2016 International Conference on Systems, Signals and Image Processing (IWSSIP).

[3]  Macario Cordel,et al.  Convolutional neural network for vehicle detection in low resolution traffic videos , 2016, 2016 IEEE Region 10 Symposium (TENSYMP).

[4]  Stephen J. Sangwine,et al.  Quaternion Fourier Transforms for Signal and Image Processing , 2014 .

[5]  Lucia Maddalena,et al.  Towards Benchmarking Scene Background Initialization , 2015, ICIAP Workshops.

[6]  Licheng Yu,et al.  Vector Sparse Representation of Color Image Using Quaternion Matrix Analysis , 2015, IEEE Transactions on Image Processing.

[7]  Thomas Peters,et al.  Data-driven science and engineering: machine learning, dynamical systems, and control , 2019, Contemporary Physics.

[8]  Fuzhen Zhang.  Quaternions and matrices of quaternions , 1997 .

[9]  Khalid M. Hosny,et al.  New set of multi-channel orthogonal moments for color image representation and recognition , 2019, Pattern Recognit..

[10]  Fahd Bouzaraa,et al.  CNN-based initial background estimation , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[11]  Steven L. Brunton,et al.  Multi-resolution Dynamic Mode Decomposition for Foreground/Background Separation and Object Tracking , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[12]  José María Martínez Sanchez,et al.  Rejection based multipath reconstruction for background estimation in video sequences with stationary objects , 2016, Comput. Vis. Image Underst..

[13]  Lucia Maddalena,et al.  Scene background initialization: A taxonomy , 2017, Pattern Recognit. Lett..

[14]  Soo-Chang Pei,et al.  Quaternion matrix singular value decomposition and its applications for color image processing , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[15]  Atsushi Shimada,et al.  Background initialization based on bidirectional analysis and consensus voting , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[16]  Marc Van Droogenbroeck,et al.  LaBGen: A method based on motion detection for generating the background of a scene , 2017, Pattern Recognit. Lett..

[17]  Zhiwen Liu,et al.  Quaternion generic Fourier descriptor for color object recognition , 2015, Pattern Recognit..

[18]  Lucia Maddalena,et al.  Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization , 2017, IEEE Transactions on Image Processing.

[19]  Soon Ki Jung,et al.  Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset , 2015, Comput. Sci. Rev..

[20]  Eduardo A. B. da Silva,et al.  Foreground Segmentation for Anomaly Detection in Surveillance Videos Using Deep Residual Networks , 2017 .

[21]  Soon Ki Jung,et al.  Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation , 2018, Neural Networks.

[22]  Patrick R. Girard,et al.  Quaternions, Clifford Algebras and Relativistic Physics , 2007 .

[23]  Soon Ki Jung,et al.  Moving Object Detection in Complex Scene Using Spatiotemporal Structured-Sparse RPCA , 2019, IEEE Transactions on Image Processing.

[24]  Yildiray Yalman,et al.  A new color image quality measure based on YUV transformation and PSNR for human vision system , 2013 .

[25]  Mario Ignacio Chacon Murguia,et al.  Temporal weighted learning model for background estimation with an automatic re-initialization stage and adaptive parameters update , 2017, Pattern Recognit. Lett..

[26]  T. J. Stonham,et al.  A system for counting people in video images using neural networks to identify the background scene , 1996, Pattern Recognit..

[27]  William Rowan Hamilton,et al.  ON QUATERNIONS, OR ON A NEW SYSTEM OF IMAGINARIES IN ALGEBRA , 1847 .

[28]  Nicolas Le Bihan,et al.  Singular value decomposition of quaternion matrices: a new tool for vector-sensor signal processing , 2004, Signal Process..

[29]  Jiankang Zhao,et al.  Robust background generation based on an effective frames selection method and an efficient background estimation procedure (FSBE) , 2019, Signal Process. Image Commun..

[30]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[31]  Steven L. Brunton,et al.  Compressed dynamic mode decomposition for background modeling , 2015, Journal of Real-Time Image Processing.

[32]  Rabah Attia,et al.  Real-time scene background initialization based on spatio-temporal neighborhood exploration , 2018, Multimedia Tools and Applications.

[33]  Lucia Maddalena,et al.  A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications , 2008, IEEE Transactions on Image Processing.

[34]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[35]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[36]  Soon Ki Jung,et al.  Background–Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering , 2017, IEEE Transactions on Image Processing.