Learning Stable Linear Dynamical Systems with the Weighted Least Square Method

Standard subspace algorithms learn Linear Dynamical Systems (LDSs) from time series with the least-square method, where the stability of the system is not naturally guaranteed. In this paper, we propose a novel approach for learning stable systems by enforcing stability directly on the least-square solutions. To this end, we first explore the spectral-radius property of the least-square transition matrix and then determine the key component that incurs the instability of the transition matrix. By multiplying the unstable component with a weight matrix on the right side, we obtain a weighted-least-square transition matrix that is further optimized to minimize the reconstruction error of the state sequence while still maintaining the stable constraint. Comparative experimental evaluations demonstrate that our proposed methods outperform the state-of-the-art methods regarding the reconstruction accuracy and the learning efficiency.

[1]  Byron Boots,et al.  Graph-Based Inverse Optimal Control for Robot Manipulation , 2015, IJCAI.

[2]  Dennis S. Bernstein,et al.  Subspace identification with guaranteed stability using constrained optimization , 2003, IEEE Trans. Autom. Control..

[3]  Nuno Vasconcelos,et al.  Layered Dynamic Textures , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  A. L. Kazakov,et al.  On Mathematical Models for Optimization Problem of Logistics Infrastructure , 2015 .

[5]  Nuno Vasconcelos,et al.  Probabilistic kernels for the classification of auto-regressive visual processes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Antoni B. Chan,et al.  A Scalable and Accurate Descriptor for Dynamic Textures Using Bag of System Trees , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  J. Maciejowski.  Guaranteed stability with subspace methods , 1995 .

[9]  René Vidal,et al.  Categorizing Dynamic Textures Using a Bag of Dynamical Systems , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Andrew W. Fitzgibbon,et al.  Shift-Invariant Dynamic Texture Recognition , 2006, ECCV.

[11]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Luc Van Gool,et al.  European conference on computer vision (ECCV) , 2006, eccv 2006.

[13]  Deli Zhao,et al.  Recognizing an Action Using Its Name: A Knowledge-Based Approach , 2016, International Journal of Computer Vision.

[14]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[15]  Ieee Transactions On Automatic Control, Vol. Ac-2'7, No. 3, June 1982 , .

[16]  René Vidal,et al.  Group action induced distances for averaging and clustering Linear Dynamical Systems with applications to the analysis of dynamic scenes , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[18]  Alexander J. Smola,et al.  Binet-Cauchy Kernels on Dynamical Systems and its Application to the Analysis of Dynamic Scenes , 2007, International Journal of Computer Vision.

[19]  Fuchun Sun,et al.  Analyzing Linear Dynamical Systems: From Modeling to Coding and Learning , 2016, ArXiv.

[20]  Bart De Moor,et al.  N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems , 1994, Autom..

[21]  René Vidal,et al.  Optical flow estimation & segmentation of multiple moving dynamic textures , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Yi Yang,et al.  DevNet: A Deep Event Network for multimedia event detection and evidence recounting , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Byron Boots,et al.  A Constraint Generation Approach to Learning Stable Linear Dynamical Systems , 2007, NIPS.

[24]  R. Shumway,et al.  AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[25]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  Deli Zhao,et al.  Sparse Coding and Dictionary Learning with Linear Dynamical Systems , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).