Nonlinear system identification using deep belief network based on PLSR

Deep learning has been successfully applied into pattern recognition due to its deep architecture and effective unsupervised learning, and deep belief network (DBN) is a popular model based on deep learning technique. In this paper, a DBN identification model based on partial least square regression (PLSR), named PLSR-DBN, is proposed for nonlinear system identification. In order to improve the identification accuracy, PLSR is introduced into the supervised fine-tuning of DBN to elimate the overfitting and local minimum resulted from gradients-based learning, and contrastive divergence (CD) algorithm is used in unsupervised pre-training. Finally, the proposed PLSR-DBN is tested on a benchmark nonlinear system. The experiment results show that the proposed PLSR-DBN has a better performance on nonlinear system identification than other similar methods.

[1]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[2]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[3]  Yan-Lin He,et al.  A robust hybrid model integrating enhanced inputs based extreme learning machine with PLSR (PLSR-EIELM) and its application to intelligent measurement. , 2015, ISA transactions.

[4]  Junfei Qiao,et al.  A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm , 2010, IEEE Transactions on Fuzzy Systems.

[5]  Hak-Keung Lam,et al.  Tuning of the structure and parameters of a neural network using an improved genetic algorithm , 2003, IEEE Trans. Neural Networks.

[6]  Wen Yu,et al.  Randomized algorithms for nonlinear system identification with deep learning modification , 2016, Inf. Sci..

[7]  Shen Furao,et al.  Forecasting exchange rate using deep belief networks and conjugate gradient method , 2015, Neurocomputing.

[8]  Meng Joo Er,et al.  A fast learning algorithm for parsimonious fuzzy neural systems , 1999, 1999 European Control Conference (ECC).

[9]  Geoffrey E. Hinton,et al.  Understanding how Deep Belief Networks perform acoustic modelling , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Dong Yu,et al.  Investigation of full-sequence training of deep belief networks for speech recognition , 2010, INTERSPEECH.

[11]  Noel Lopes,et al.  Towards adaptive learning with improved convergence of deep belief networks on graphics processing units , 2014, Pattern Recognit..

[12]  Long Cheng,et al.  Adaptive neural network tracking control for manipulators with uncertain kinematics, dynamics and actuator model , 2009, Autom..

[13]  Frank L. Lewis,et al.  Identification of nonlinear dynamical systems using multilayered neural networks , 1996, Autom..

[14]  Andrew Rawson,et al.  The prediction of soil chemical and physical properties from mid-infrared spectroscopy and combined partial least-squares regression and neural networks (PLS-NN) analysis , 2009 .

[15]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[16]  Junfei Qiao,et al.  Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach , 2012, Autom..

[17]  Xiaoou Li,et al.  Nonlinear system identification using deep learning and randomized algorithms , 2015, 2015 IEEE International Conference on Information and Automation.

[18]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[19]  Junfei Qiao,et al.  Efficient self-organizing multilayer neural network for nonlinear system modeling , 2013, Neural Networks.

[20]  Xiaoou Li,et al.  Automated nonlinear system modelling with multiple neural networks , 2011, Int. J. Syst. Sci..

[21]  Meng Joo Er,et al.  Self-constructing Fuzzy Neural Networks with Extended Kalman Filter , 2010 .

[22]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[23]  Nicolas Le Roux,et al.  Representational Power of Restricted Boltzmann Machines and Deep Belief Networks , 2008, Neural Computation.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[26]  Xiao Li,et al.  Machine Learning Paradigms for Speech Recognition: An Overview , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[27]  Naif Alajlan,et al.  Deep learning approach for active classification of electrocardiogram signals , 2016, Inf. Sci..

[28]  Junfei Qiao,et al.  Nonlinear Systems Modeling Based on Self-Organizing Fuzzy-Neural-Network With Adaptive Computation Algorithm , 2014, IEEE Transactions on Cybernetics.