Hybrid system identification using a mixture of NARX experts with LASSO-based feature selection

The availability of advanced hybrid system identification techniques is fundamental to extract knowledge in form of models from data streams. Starting from the current state of the art, we propose an approach based on a specialized architecture, conceived to address the peculiar integration of nonlinear dynamics and finite state switching behavior of hybrid systems. Following the Mixtures of Experts concept, we combine a set of Neural Network ARX (NNARX) models with a Gated Recurrent Units network with softmax output. The former are exploited to map specific nonlinear dynamical models representing the behavior of the system in each discrete mode of operation. The latter, operating as a neural switching machine, infers the unobserved active mode and learns the state-transition logic, conditioned on input-output data sequences. Besides, we integrate a LASSO based input features and model selection mechanism, aimed to extract the most informative lags over the sequences for each NNARX and calibrate the modes to be employed. The overall system is trained end-to-end. Experiments have been performed on a benchmark hybrid automata with nonlinear dynamics and transitions, showing the capability to achieve improved performances than conventional architectures.

[1]  Yoshua Bengio,et al.  An Input Output HMM Architecture , 1994, NIPS.

[2]  Laurent Bako,et al.  Identification of switched linear systems via sparse optimization , 2011, Autom..

[3]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[4]  Hien D. Nguyen,et al.  Regularized Estimation and Feature Selection in Mixtures of Gaussian-Gated Experts Models , 2019, Communications in Computer and Information Science.

[5]  L. Ljung,et al.  Segmentation of Time Series from Nonlinear Dynamical Systems , 2011 .

[6]  Alberto Bemporad,et al.  Piecewise affine regression via recursive multiple least squares and multicategory discrimination , 2016, Autom..

[7]  Danilo Comminiello,et al.  Group sparse regularization for deep neural networks , 2016, Neurocomputing.

[8]  JacobssonHenrik.  Rule Extraction from Recurrent Neural Networks: A Taxonomy and Review , 2005 .

[9]  Ieroham S. Baruch,et al.  Hybrid Recurrent Neural Network for Nonlinear Hybrid Dynamical Systems Identification , 2011, 2011 8th International Conference on Electrical Engineering, Computing Science and Automatic Control.

[10]  Gérard Bloch,et al.  Reduced-Size Kernel Models for Nonlinear Hybrid System Identification , 2011, IEEE Transactions on Neural Networks.

[11]  Matteo Matteucci,et al.  Nonlinear system identification using a recurrent network in a Bayesian framework , 2019, 2019 IEEE 17th International Conference on Industrial Informatics (INDIN).

[12]  Wei Zhou,et al.  Data driven discovery of cyber physical systems , 2018, Nature Communications.

[13]  Hod Lipson,et al.  Learning symbolic representations of hybrid dynamical systems , 2012, J. Mach. Learn. Res..

[14]  Yordan Hristov,et al.  Hybrid system identification using switching density networks , 2019, CoRL.

[15]  Ashok N. Srivastava,et al.  Nonlinear gated experts for time series: discovering regimes and avoiding overfitting , 1995, Int. J. Neural Syst..

[16]  S. L. Brunton,et al.  Model selection for hybrid dynamical systems via sparse regression , 2018, Proceedings of the Royal Society A.

[17]  Joseph N. Wilson,et al.  Twenty Years of Mixture of Experts , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Jan Lunze,et al.  Handbook of hybrid systems control : theory, tools, applications , 2009 .

[19]  Steven J. Nowlan,et al.  Mixtures of Controllers for Jump Linear and Non-Linear Plants , 1993, NIPS.

[20]  Bernard Riera,et al.  Using Neural Networks for the identification of a class of Hybrid Dynamic Systems , 2006, ADHS.

[21]  Alvaro Soto,et al.  Embedded local feature selection within mixture of experts , 2014, Inf. Sci..

[22]  Gérard Bloch,et al.  Learning nonlinear hybrid systems: from sparse optimization to support vector regression , 2013, HSCC '13.

[23]  A. Garulli,et al.  A survey on switched and piecewise affine system identification , 2012 .

[24]  Lorenzo Fagiano,et al.  Day ahead electricity price forecast by NARX model with LASSO based features selection , 2019, 2019 IEEE 17th International Conference on Industrial Informatics (INDIN).

[25]  Gérard Bloch,et al.  Switched and PieceWise Nonlinear Hybrid System Identification , 2008, HSCC.