Streaming GPU Singular Value and Dynamic Mode Decompositions

This work develops a parallelized algorithm to compute the dynamic mode decomposition (DMD) on a graphics processing unit using the streaming method of snapshots singular value decomposition. This allows the algorithm to operate efficiently on streaming data by avoiding redundant inner-products as new data becomes available. In addition, it is possible to leverage the native compressed format of many data streams, such as HD video and computational physics codes that are represented sparsely in the Fourier domain, to massively reduce data transfer from CPU to GPU and to enable sparse matrix multiplications. Taken together, these algorithms facilitate real-time streaming DMD on high-dimensional data streams. We demonstrate the proposed method on numerous high-dimensional data sets ranging from video background modeling to scientific computing applications, where DMD is becoming a mainstay algorithm. The computational framework is developed as an open-source library written in C++ with CUDA, and the algorithms may be generalized to include other DMD advances, such as compressed sensing DMD, multi resolution DMD, or DMD with control. Keywords: Singular value decomposition, dynamic mode decomposition, streaming computations, graphics processing unit, video background modeling, scientific computing.

[1]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[2]  Uri Shaham,et al.  Dynamic Mode Decomposition , 2013 .

[3]  Qian Wang,et al.  AUGEM: Automatically generate high performance Dense Linear Algebra kernels on x86 CPUs , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[4]  El-hadi Zahzah,et al.  Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing , 2016 .

[5]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[6]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[7]  Touradj Ebrahimi,et al.  UHD video dataset for evaluation of privacy , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[8]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[9]  N. Benjamin Erichson,et al.  Randomized low-rank Dynamic Mode Decomposition for motion detection , 2015, Comput. Vis. Image Underst..

[10]  D. A. Bistrian,et al.  Randomized dynamic mode decomposition for nonintrusive reduced order modelling , 2016, 1611.04884.

[11]  Thierry Chateau,et al.  A Benchmark Dataset for Outdoor Foreground/Background Extraction , 2012, ACCV Workshops.

[12]  W. Stankiewicz,et al.  Recursive dynamic mode decomposition of a transient cylinder wake , 2015, 1511.06876.

[13]  WeberRick,et al.  From CUDA to OpenCL , 2012 .

[14]  Heni Ben Amor,et al.  Dynamic Mode Decomposition for perturbation estimation in human robot interaction , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[15]  J. Nathan Kutz,et al.  Dynamic Mode Decomposition for Real-Time Background/Foreground Separation in Video , 2014, ArXiv.

[16]  Peter J. Schmid,et al.  Parallel data-driven decomposition algorithm for large-scale datasets: with application to transitional boundary layers , 2016 .

[17]  Peter J. Schmid,et al.  Sparsity-promoting dynamic mode decomposition , 2012, 1309.4165.

[18]  S. Osher,et al.  Sparse dynamics for partial differential equations , 2012, Proceedings of the National Academy of Sciences.

[19]  Jong-Il Park,et al.  Computer Vision - ACCV 2012 Workshops , 2012, Lecture Notes in Computer Science.

[20]  L. Sirovich.  TURBULENCE AND THE DYNAMICS OF COHERENT STRUCTURES PART I : COHERENT STRUCTURES , 2016 .

[21]  Stanley Osher,et al.  On the Compressive Spectral Method , 2014, Multiscale Model. Simul..

[22]  B. O. Koopman,et al.  Hamiltonian Systems and Transformation in Hilbert Space. , 1931, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Steven L. Brunton,et al.  Multiresolution Dynamic Mode Decomposition , 2015, SIAM J. Appl. Dyn. Syst..

[24]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[25]  L. Sirovich.  Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[26]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[27]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[28]  Jack J. Dongarra,et al.  Towards dense linear algebra for hybrid GPU accelerated manycore systems , 2009, Parallel Comput..

[29]  B. R. Noack,et al.  Closed-Loop Turbulence Control: Progress and Challenges , 2015 .

[30]  Steven L. Brunton,et al.  Compressed sensing and dynamic mode decomposition , 2016 .

[31]  Jack J. Dongarra,et al.  Dense linear algebra solvers for multicore with GPU accelerators , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[32]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[33]  David Windridge,et al.  Can DMD obtain a Scene Background in color? , 2016, 2016 International Conference on Image, Vision and Computing (ICIVC).

[34]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[35]  Steven L. Brunton,et al.  Compressed Dynamic Mode Decomposition for Real-Time Object Detection , 2015, ArXiv.

[36]  Thierry Bouwmans,et al.  Traditional and recent approaches in background modeling for foreground detection: An overview , 2014, Comput. Sci. Rev..

[37]  Peter Läuchli,et al.  Jordan-Elimination und Ausgleichung nach kleinsten Quadraten , 1961 .

[38]  P. Schmid,et al.  Applications of the dynamic mode decomposition , 2011 .

[39]  Jack J. Dongarra,et al.  From CUDA to OpenCL: Towards a performance-portable solution for multi-platform GPU programming , 2012, Parallel Comput..

[40]  Joshua L. Proctor,et al.  Discovering dynamic patterns from infectious disease data using dynamic mode decomposition , 2015, International health.

[41]  Ionel Michael Navon,et al.  On linear and nonlinear aspects of dynamic mode decomposition , 2016 .

[42]  I. Mezić,et al.  Analysis of Fluid Flows via Spectral Properties of the Koopman Operator , 2013 .

[43]  Clarence W. Rowley,et al.  Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition , 2014, Experiments in Fluids.

[44]  Zhu Wang,et al.  Approximate partitioned method of snapshots for POD , 2016, J. Comput. Appl. Math..

[45]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[46]  Clarence W. Rowley,et al.  Low-frequency dynamics in a shock-induced separated flow , 2016, Journal of Fluid Mechanics.

[47]  I. Mezić.  Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[48]  Firas Hamze,et al.  A Performance Comparison of CUDA and OpenCL , 2010, ArXiv.

[49]  Matthew Brand,et al.  Incremental Singular Value Decomposition of Uncertain Data with Missing Values , 2002, ECCV.

[50]  Clarence W. Rowley,et al.  Dynamic mode decomposition for large and streaming datasets , 2014, 1406.7187.

[51]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[52]  Bingni W. Brunton,et al.  Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition , 2014, Journal of Neuroscience Methods.

[53]  Steven L. Brunton,et al.  Compressed dynamic mode decomposition for background modeling , 2015, Journal of Real-Time Image Processing.

[54]  Steven L. Brunton,et al.  Dynamic mode decomposition - data-driven modeling of complex systems , 2016 .