An Algebraic Translation of Cayley-Dickson Linear Systems and Its Applications to Online Learning

The m-dimensional Cayley-Dickson number system Am is a standard extension of real (m=1), complex (m=2), quaternion (m=22), octonion (m=23) and sedenion (m=24) etc. In this paper, we present a systematic algebraic translation of the Cayley-Dickson hypercomplex valued linear systems into a real vector valued linear model. This translation is designed by using jointly two new isomorphisms between real vector spaces and enables us to straightforwardly apply the well established schemes in real domain to problems for the hypercomplex linear model. We also clarify useful algebraic properties of the proposed translation. As an example of many potential algorithms through the proposed algebraic translation, we present Am-adaptive projected subgradient method ( Am-APSM) for Am valued adaptive system identification, and show that many hypercomplex adaptive filtering algorithms can be viewed as special cases of this algorithm. Moreover, we also apply the Am-APSM to nonlinear adaptive filtering by using the kernel trick. Numerical examples show that the effectiveness of the Am-APSM in many Cayley-Dickson valued linear system identification and nonlinear channel equalization problems.

[1]  Kazuyuki Aihara,et al.  Quaternion-valued short term forecasting of wind profile , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[2]  A. Hjørungnes.  Complex-Valued Matrix Derivatives: Applications in Signal Processing and Communications , 2011 .

[3]  Danilo P. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters , 2009 .

[4]  John C. Baez,et al.  The Octonions , 2001 .

[5]  R. D. Schafer.  An Introduction to Nonassociative Algebras , 1966 .

[6]  B. A. D. H. Brandwood.  A complex gradient operator and its applica-tion in adaptive array theory , 1983 .

[7]  Danilo P. Mandic,et al.  On gradient calculation in quaternion adaptive filtering , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[9]  Y. Engel.  Kernel Recursive Least Squares , 2004 .

[10]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[11]  A. S. Solodovnikov,et al.  Hypercomplex Numbers: An Elementary Introduction to Algebras , 1989 .

[12]  Clifford,et al.  Applications of Grassmann's Extensive Algebra , 1878 .

[13]  A. Bos.  Complex gradient and Hessian , 1994 .

[14]  Sergios Theodoridis,et al.  Ieee Transactions on Signal Processing Extension of Wirtinger's Calculus to Reproducing Kernel Hilbert Spaces and the Complex Kernel Lms , 2022 .

[15]  Sergios Theodoridis,et al.  Adaptive Learning in Complex Reproducing Kernel Hilbert Spaces Employing Wirtinger's Subgradients , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[16]  J.C.K. Chou,et al.  Quaternion kinematic and dynamic differential equations , 1992, IEEE Trans. Robotics Autom..

[17]  Jacob Benesty,et al.  Acoustic signal processing for telecommunication , 2000 .

[18]  Tülay Adali,et al.  Complex-Valued Signal Processing: The Proper Way to Deal With Impropriety , 2011, IEEE Transactions on Signal Processing.

[19]  Danilo P. Mandic,et al.  A Quaternion Widely Linear Adaptive Filter , 2010, IEEE Transactions on Signal Processing.

[20]  Isao Yamada,et al.  The Adaptive Projected Subgradient Method Constrained by Families of Quasi-nonexpansive Mappings and Its Application to Online Learning , 2010, SIAM J. Optim..

[21]  Isao Yamada,et al.  An algebraic real translation of hypercomplex linear systems and its application to adaptive filtering , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  D. Alfsmann.  On families of 2 N -dimensional hypercomplex algebras suitable for digital signal processing , 2006 .

[23]  Kazuhiko Ozeki,et al.  An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties , 1984 .

[24]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[25]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[26]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[27]  Danilo P. Mandic,et al.  A Quaternion Gradient Operator and Its Applications , 2011, IEEE Signal Processing Letters.

[28]  Danilo P. Mandic,et al.  A class of quaternion valued affine projection algorithms , 2013, Signal Process..

[29]  Danilo P. Mandic,et al.  An augmented affine projection algorithm for the filtering of noncircular complex signals , 2010, Signal Process..

[30]  Simon Haykin,et al.  Adaptive Signal Processing: Next Generation Solutions , 2010 .

[31]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[32]  Paulo S. R. Diniz,et al.  Adaptive Filtering: Algorithms and Practical Implementation , 1997 .

[33]  Danilo P. Mandic,et al.  Quaternion-Valued Nonlinear Adaptive Filtering , 2011, IEEE Transactions on Neural Networks.

[34]  Pascal Chevalier,et al.  Widely linear estimation with complex data , 1995, IEEE Trans. Signal Process..

[35]  Kazuyuki Aihara,et al.  Complex-valued forecasting of wind profile , 2006 .

[36]  B. Widrow,et al.  The complex LMS algorithm , 1975, Proceedings of the IEEE.

[37]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[38]  V. Paulsen,et al.  An Introduction to the Theory of Reproducing Kernel Hilbert Spaces , 2016 .

[39]  Vitor H. Nascimento,et al.  Second-order analysis of the RC-WL-QLMS algorithm , 2012, 2012 International Symposium on Wireless Communication Systems (ISWCS).

[40]  Jacob Benesty,et al.  A Widely Linear Distortionless Filter for Single-Channel Noise Reduction , 2010, IEEE Signal Processing Letters.

[41]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[42]  S. Haykin,et al.  Kernel Least‐Mean‐Square Algorithm , 2010 .

[43]  Hualiang Li,et al.  On properties of the widely linear MSE filter and its LMS implementation , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[44]  L. Scharf,et al.  Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular Signals , 2010 .

[45]  T. Hinamoto,et al.  Extended theory of learning identification , 1975 .

[46]  Julian J. Faraway,et al.  Modeling Head and Hand Orientation during Motion using Quaternions , 2004 .

[47]  Vitor H. Nascimento,et al.  A novel reduced-complexity widely linear QLMS algorithm , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[48]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[49]  Isao Yamada,et al.  An efficient robust adaptive filtering algorithm based on parallel subgradient projection techniques , 2002, IEEE Trans. Signal Process..

[50]  J. Nagumo,et al.  A learning method for system identification , 1967, IEEE Transactions on Automatic Control.

[51]  Youngchul Sung,et al.  Dual-Domain Adaptive Beamformer Under Linearly and Quadratically Constrained Minimum Variance , 2013, IEEE Transactions on Signal Processing.

[52]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[53]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[54]  Paul Honeine,et al.  Online Prediction of Time Series Data With Kernels , 2009, IEEE Transactions on Signal Processing.

[55]  Sergios Theodoridis,et al.  Stochastic Analysis of Hyperslab-Based Adaptive Projected Subgradient Method Under Bounded Noise , 2013, IEEE Signal Processing Letters.

[56]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[57]  W. Wirtinger.  Zur formalen Theorie der Funktionen von mehr komplexen Veränderlichen , 1927 .

[58]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.