Basis Function Matrix-Based Flexible Coefficient Autoregressive Models: A Framework for Time Series and Nonlinear System Modeling

We propose, in this paper, a framework for time series and nonlinear system modeling, called the basis function matrix-based flexible coefficient autoregressive (BFM-FCAR) model. It has very flexible nonlinear structure. We show that many famous nonlinear time series models can be derived under this framework by choosing the proper basis function matrices. Some probabilistic properties (the conditions of geometrical ergodicity) of the BFM-FCAR model are investigated. Taking advantage of the model structure, we present an efficient parameter estimation algorithm for the proposed framework by using the variable projection method. Finally, we show how new models are generated from the proposed framework.

[1]  Min Gan,et al.  An Efficient Variable Projection Formulation for Separable Nonlinear Least Squares Problems , 2014, IEEE Transactions on Cybernetics.

[2]  H. An,et al.  The geometrical ergodicity of nonlinear autoregressive models , 1996 .

[3]  H. Tong,et al.  On the use of the deterministic Lyapunov function for the ergodicity of stochastic difference equations , 1985, Advances in Applied Probability.

[4]  Feng Ding,et al.  Highly Efficient Identification Methods for Dual-Rate Hammerstein Systems , 2015, IEEE Transactions on Control Systems Technology.

[5]  T. Ozaki,et al.  Modelling nonlinear random vibrations using an amplitude-dependent autoregressive time series model , 1981 .

[6]  Feng Ding,et al.  A multi-innovation state and parameter estimation algorithm for a state space system with d-step state-delay , 2017, Signal Process..

[7]  Zhi Liu,et al.  A Probabilistic Neural-Fuzzy Learning System for Stochastic Modeling , 2008, IEEE Transactions on Fuzzy Systems.

[8]  Min Gan,et al.  Generalized exponential autoregressive models for nonlinear time series: Stationarity, estimation and applications , 2018, Inf. Sci..

[9]  Jeffrey S. Racine,et al.  Semiparametric ARX neural-network models with an application to forecasting inflation , 2001, IEEE Trans. Neural Networks.

[10]  Marcelo C. Medeiros,et al.  A flexible coefficient smooth transition time series model , 2005, IEEE Transactions on Neural Networks.

[11]  Dongqing Wang,et al.  Model recovery for Hammerstein systems using the auxiliary model based orthogonal matching pursuit method , 2018 .

[12]  Min Gan,et al.  A Variable Projection Approach for Efficient Estimation of RBF-ARX Model , 2015, IEEE Transactions on Cybernetics.

[13]  Han-Xiong Li,et al.  Probabilistic Fuzzy Classification for Stochastic Data , 2017, IEEE Transactions on Fuzzy Systems.

[14]  T. Ozaki,et al.  Non-linear time series models for non-linear random vibrations , 1980, Journal of Applied Probability.

[15]  Gene H. Golub,et al.  The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate , 1972, Milestones in Matrix Computation.

[16]  C. L. Philip Chen,et al.  A Regularized Variable Projection Algorithm for Separable Nonlinear Least-Squares Problems , 2019, IEEE Transactions on Automatic Control.

[17]  Kurt Hornik,et al.  Stationary and Integrated Autoregressive Neural Network Processes , 2000, Neural Computation.

[18]  Shuo Zhang,et al.  Instrumental Variable-Based OMP Identification Algorithm for Hammerstein Systems , 2018, Complex..

[19]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[20]  Marcelo C. Medeiros,et al.  A hybrid linear-neural model for time series forecasting , 2000, IEEE Trans. Neural Networks Learn. Syst..

[21]  Long Chen,et al.  On Some Separated Algorithms for Separable Nonlinear Least Squares Problems , 2018, IEEE Transactions on Cybernetics.

[22]  Hannu T. Toivonen,et al.  State-dependent parameter modelling and identification of stochastic non-linear sampled-data systems , 2006 .

[23]  C. E. Pedreira,et al.  Local-global neural networks: a new approach for nonlinear time series modelling , 2003 .

[24]  R. Tweedie.  Criteria for classifying general Markov chains , 1976, Advances in Applied Probability.

[25]  M. Viberg,et al.  Separable non-linear least-squares minimization-possible improvements for neural net fitting , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[26]  D. Tjøstheim.  Non-linear time series and Markov chains , 1990, Advances in Applied Probability.

[27]  H. Tong,et al.  From patterns to processes: phase and density dependencies in the Canadian lynx cycle. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Shaocheng Tong,et al.  Fuzzy Adaptive Decentralized Optimal Control for Strict Feedback Nonlinear Large-Scale Systems , 2018, IEEE Transactions on Cybernetics.

[29]  Dongqing Wang,et al.  Decoupled Parameter Estimation Methods for Hammerstein Systems by Using Filtering Technique , 2018, IEEE Access.

[30]  M. Priestley.  STATE‐DEPENDENT MODELS: A GENERAL APPROACH TO NON‐LINEAR TIME SERIES ANALYSIS , 1980 .

[31]  Ruey S. Tsay,et al.  Functional-Coefficient Autoregressive Models , 1993 .

[32]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[33]  Feng Ding,et al.  Combined state and parameter estimation for a bilinear state space system with moving average noise , 2018, J. Frankl. Inst..

[34]  José Manuel Benítez,et al.  Equivalences Between Neural-Autoregressive Time Series Models and Fuzzy Systems , 2010, IEEE Transactions on Neural Networks.

[35]  T. Hayat,et al.  Parameter estimation for pseudo-linear systems using the auxiliary model and the decomposition technique , 2017 .

[36]  T. Teräsvirta.  Specification, Estimation, and Evaluation of Smooth Transition Autoregressive Models , 1994 .

[37]  Aleksandr Y. Aravkin,et al.  Estimating nuisance parameters in inverse problems , 2012, 1206.6532.

[38]  Torsten Söderström,et al.  Fault Detection of Nonlinear Systems by Using Hybrid Quasi-ARMAX Models , 1997 .

[39]  Erfu Yang,et al.  State filtering‐based least squares parameter estimation for bilinear systems using the hierarchical identification principle , 2018, IET Control Theory & Applications.

[40]  Yanjun Liu,et al.  Model recovery for Hammerstein systems using the hierarchical orthogonal matching pursuit method , 2019, J. Comput. Appl. Math..

[41]  James V. Burke,et al.  Algorithmic Differentiation of Implicit Functions and Optimal Values , 2008 .

[42]  C. L. Philip Chen,et al.  Gradient Radial Basis Function Based Varying-Coefficient Autoregressive Model for Nonlinear and Nonstationary Time Series , 2015, IEEE Signal Processing Letters.

[43]  Jean-Marc Vesin,et al.  An amplitude-dependent autoregressive model based on a radial basis functions expansion , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[44]  Robert Haber.  Nonlinear System Identification : Input-output Modeling Approach , 1999 .

[45]  K. Hirasawa,et al.  A Quasi-ARMAX approach to modelling of non-linear systems , 2001 .

[46]  C. L. Philip Chen,et al.  Exploiting the interpretability and forecasting ability of the RBF-AR model for nonlinear time series , 2016, Int. J. Syst. Sci..

[47]  Tohru Ozaki,et al.  RBF-ARX MODELING FOR PREDICTION AND CONTROL , 2006 .

[48]  Min Gan,et al.  A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling , 2010, Inf. Sci..

[49]  Timo Teräsvirta,et al.  Forecasting economic variables with nonlinear models , 2005 .

[50]  Kazushi Nakano,et al.  Nonlinear Predictive Control Using Neural Nets-Based Local Linearization ARX Model—Stability and Industrial Application , 2007, IEEE Transactions on Control Systems Technology.

[51]  Fuad E. Alsaadi,et al.  Iterative parameter identification for pseudo-linear systems with ARMA noise using the filtering technique , 2018 .

[52]  Yukihiro Toyoda,et al.  An Akaike State-Space Controller for RBF-ARX Models , 2009, IEEE Transactions on Control Systems Technology.