Non-Linear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms

New learning algorithms for an adaptive non-linear forward predictor which is based on a Pipelined Recurrent Neural Network (PRNN) are presented. A computationally efficient Gradient Descent (GD) algorithm, as well as a novel Extended Recursive Least Squares (ERLS) algorithm are tested on the predictor. Simulation studies, based on three speech signals, which have been made public and are available on the World Wide Web (WWW), show that the non-linear predictor does not perform satisfactorily when the previously proposed gradient descent algorithm was used. The steepest descent algorithm is shown to yield a poor performance in terms of the prediction error gain, whereas consistently improved results are obtained using the ERLS algorithm. The merit of the non-linear predictor structure is confirmed by yielding approximately 2 dB higher prediction gain than only a linear structure predictor, which uses the conventional Recursive Least Squares (RLS) algorithm.

[1]  J. Makhoul,et al.  Linear prediction: A tutorial review , 1975, Proceedings of the IEEE.

[2]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[3]  J. Shynk.  Adaptive IIR filtering , 1989, IEEE ASSP Magazine.

[4]  C. N. Manikopoulos,et al.  Neural net nonlinear prediction for speech data , 1991 .

[5]  Visakan Kadirkamanathan,et al.  A nonlinear model for time series prediction and signal interpolation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[6]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[7]  Kumpati S. Narendra,et al.  Neural Networks In Dynamical Systems , 1990, Other Conferences.

[8]  C. P. Sheppard,et al.  Predicting time series by a fully connected neural network trained by back propagation , 1992 .

[9]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[10]  Yoshua Bengio,et al.  Neural networks for speech and sequence recognition , 1996 .

[11]  Pierre Roussel-Ragot,et al.  Training recurrent neural networks: why and how? An illustration in dynamical process modeling , 1994, IEEE Trans. Neural Networks.

[12]  S Z Qin,et al.  Comparison of four neural net learning methods for dynamic system identification , 1992, IEEE Trans. Neural Networks.

[13]  Liang Li,et al.  Nonlinear adaptive prediction of nonstationary signals , 1995, IEEE Trans. Signal Process..

[14]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[15]  Sailes K. Sengijpta.  Fundamentals of Statistical Signal Processing: Estimation Theory , 1995 .

[16]  B. Townshend,et al.  Nonlinear prediction of speech , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[17]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[18]  S. Haykin,et al.  A cascaded recurrent neural network for real-time nonlinear adaptive filtering , 1993, IEEE International Conference on Neural Networks.

[19]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .