Scaled Conjugate Gradient Learning for Quaternion-Valued Neural Networks

This paper presents the deduction of the scaled conjugate gradient method for training quaternion-valued feedforward neural networks, using the framework of the HR calculus. The performances of the scaled conjugate algorithm in the real- and complex-valued cases lead to the idea of extending it to the quaternion domain, also. Experiments done using the proposed training method on time series prediction applications showed a significant performance improvement over the quaternion gradient descent and quaternion conjugate gradient algorithms.

[1]  Danilo P. Mandic,et al.  The Quaternion LMS Algorithm for Adaptive Filtering of Hypercomplex Processes , 2009, IEEE Transactions on Signal Processing.

[2]  Danilo P. Mandic,et al.  On HR calculus, quaternion valued stochastic gradient, and adaptive three dimensional wind forecasting , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[3]  Danilo P. Mandic,et al.  A Quaternion Widely Linear Adaptive Filter , 2010, IEEE Transactions on Signal Processing.

[4]  Giovanni Muscato,et al.  Multilayer Perceptrons to Approximate Quaternion Valued Functions , 1997, Neural Networks.

[5]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[6]  Kazuyuki Aihara,et al.  Quaternion-valued short term forecasting of wind profile , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[7]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[8]  Nicolas Le Bihan,et al.  Polarized Signal Classification by Complex and quaternionic Multi-Layer Perceptrons , 2008, Int. J. Neural Syst..

[9]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[10]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[11]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[12]  Calin-Adrian Popa,et al.  Scaled Conjugate Gradient Learning for Complex-Valued Neural Networks , 2015, MENDEL.

[13]  Danilo P. Mandic,et al.  Quaternion-Valued Nonlinear Adaptive Filtering , 2011, IEEE Transactions on Neural Networks.

[14]  Danilo P. Mandic,et al.  Quaternion-Valued Stochastic Gradient-Based Adaptive IIR Filtering , 2010, IEEE Transactions on Signal Processing.

[15]  Danilo P. Mandic,et al.  Quaternion-Valued Echo State Networks , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Danilo P. Mandic,et al.  Optimization in Quaternion Dynamic Systems: Gradient, Hessian, and Learning Algorithms , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[17]  P. Arena,et al.  Neural Networks in Multidimensional Domains: Fundamentals and New Trends in Modelling and Control , 1998 .

[18]  Danilo P. Mandic,et al.  Split quaternion nonlinear adaptive filtering , 2010, Neural Networks.

[19]  Farid U. Dowla,et al.  Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method , 1991, Int. J. Neural Syst..

[20]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[21]  Kazuyuki Aihara,et al.  Quaternion-valued short term joint forecasting of three-dimensional wind and atmospheric parameters , 2011 .

[22]  Nobuyuki Matsui,et al.  Quaternion Neural Network and Its Application , 2003, KES.

[23]  P. Arena,et al.  Neural networks in multidimensional domains , 1998 .

[24]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction , 2001 .

[25]  Hiromi Kusamichi,et al.  A New Scheme for Color Night Vision by Quaternion Neural Network , 2004 .