Quaternion-Valued Nonlinear Adaptive Filtering

A class of nonlinear quaternion-valued adaptive filtering algorithms is proposed based on locally analytic nonlinear activation functions. To circumvent the stringent standard analyticity conditions which are prohibitive to the development of nonlinear adaptive quaternion-valued estimation models, we use the fact that stochastic gradient learning algorithms require only local analyticity at the operating point in the estimation space. It is shown that the quaternion-valued exponential function is locally analytic, and, since local analyticity extends to polynomials, products, and ratios, we show that a class of transcendental nonlinear functions can serve as activation functions in nonlinear and neural adaptive models. This provides a unifying framework for the derivation of gradient-based learning algorithms in the quaternion domain, and the derived algorithms are shown to have the same generic form as their real- and complex-valued counterparts. To make such models second-order optimal for the generality of quaternion signals (both circular and noncircular), we use recent developments in augmented quaternion statistics to introduce widely linear versions of the proposed nonlinear adaptive quaternion valued filters. This allows full exploitation of second-order information in the data, contained both in the covariance and pseudocovariances to cater rigorously for second-order noncircularity (improperness), and the corresponding power mismatch in the signal components. Simulations over a range of circular and noncircular synthetic processes and a real world 3-D noncircular wind signal support the approach.

[1]  Daesik Hong,et al.  Nonlinear blind equalization schemes using complex-valued multilayer feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[2]  E. Stiefel,et al.  On Cauchy-Riemann equations in higher dimensions , 1952 .

[3]  Pascal Chevalier,et al.  Widely linear estimation with complex data , 1995, IEEE Trans. Signal Process..

[4]  Kazuyuki Aihara,et al.  Complex-valued forecasting of wind profile , 2006 .

[5]  Nicolas Le Bihan,et al.  Polarized Signal Classification by Complex and quaternionic Multi-Layer Perceptrons , 2008, Int. J. Neural Syst..

[6]  T. Nitta A backpropagation algorithm for neural networks based an 3D vector product , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[7]  Julian J. Faraway,et al.  Modeling Head and Hand Orientation during Motion using Quaternions , 2004 .

[8]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[9]  Ignacio Santamaría,et al.  Properness and Widely Linear Processing of Quaternion Random Vectors , 2010, IEEE Transactions on Information Theory.

[10]  I. Bar-Itzhack,et al.  Novel quaternion Kalman filter , 2002, IEEE Transactions on Aerospace and Electronic Systems.

[11]  Andrew Chang Quaternionic analysis , 1979, Mathematical Proceedings of the Cambridge Philosophical Society.

[12]  Daniel Pérez Palomar,et al.  Generalized Likelihood Ratios for Testing the Properness of Quaternion Gaussian Vectors , 2011, IEEE Transactions on Signal Processing.

[13]  N. L. Bihan,et al.  Quaternionic Independent Component Analysis using hypercomplex nonlinearities , 2006 .

[14]  Danilo P. Mandic,et al.  The Quaternion LMS Algorithm for Adaptive Filtering of Hypercomplex Processes , 2009, IEEE Transactions on Signal Processing.

[15]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons , 2000 .

[16]  Tülay Adali,et al.  Approximation by Fully Complex Multilayer Perceptrons , 2003, Neural Computation.

[17]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction , 2001 .

[18]  Claudio Moraga,et al.  Multilayer Feedforward Neural Network Based on Multi-valued Neurons (MLMVN) and a Backpropagation Learning Algorithm , 2006, Soft Comput..

[19]  P. Arena,et al.  Neural networks in multidimensional domains , 1998 .

[20]  Norbert Jankowski,et al.  Survey of Neural Transfer Functions , 1999 .

[21]  Akira Hirose,et al.  Complex-Valued Neural Networks: Theories and Applications , 2003 .

[22]  Danilo P. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters , 2009 .

[23]  Javier Calpe-Maravilla,et al.  An easy demonstration of the optimum value of the adaptation constant in the LMS algorithm [FIR filt , 1998 .

[24]  J.C.K. Chou,et al.  Quaternion kinematic and dynamic differential equations , 1992, IEEE Trans. Robotics Autom..

[25]  Danilo P. Mandic,et al.  Augmented second-order statistics of quaternion random signals , 2011, Signal Process..

[26]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[27]  Bernard C. Picinbono,et al.  On circularity , 1994, IEEE Trans. Signal Process..

[28]  N. L. Bihan On Properness Of Quaternion Valued Random Variables , 2006 .

[29]  Danilo P. Mandic,et al.  A Quaternion Gradient Operator and Its Applications , 2011, IEEE Signal Processing Letters.

[30]  James L. Massey,et al.  Proper complex random processes with applications to information theory , 1993, IEEE Trans. Inf. Theory.

[31]  Lilong Shi EXPLORATION IN QUATERNION COLOUR , 2005 .

[32]  Danilo P. Mandic,et al.  Split quaternion nonlinear adaptive filtering , 2010, Neural Networks.

[33]  Akira Hirose,et al.  Proposal of relative-minimization learning for behavior stabilization of complex-valued recurrent neural networks , 1999, Neurocomputing.

[34]  N. Vakhania RANDOM VECTORS WITH VALUES IN QUATERNION HILBERT SPACES , 1999 .

[35]  R. Watson The Generalized Cauchy-Riemann-Fueter Equation and Handedness , 2003 .

[36]  C. Deavours The Quaternion Calculus , 1973 .

[37]  Kazuyuki Aihara,et al.  Complex-valued prediction of wind profile using augmented complex statistics , 2009 .

[38]  Daniel Pérez Palomar,et al.  Quaternion ICA From Second-Order Statistics , 2011, IEEE Transactions on Signal Processing.

[39]  Danilo P. Mandic,et al.  A Quaternion Widely Linear Adaptive Filter , 2010, IEEE Transactions on Signal Processing.

[40]  Toshimichi Saito,et al.  Torus doubling and hyperchaos in a five dimensional hysteresis circuit , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[41]  F. F. Brackx The exponential function of a quaternion variable , 1979 .