Contravariant Adaptation on the Manifold of Invertible Matrix Transfer Functions

Adaptation using contravariant, sometimes called the natural gradient, provides true steepest descent learning on a manifold such as the manifold of invertible matrices or the manifold of invertible matrix transfer functions. In this letter, the contravariant gradient on the manifold of invertible matrix transfer functions is presented using the language of differential geometry. The covariant gradient (the usual gradient) is converted to the contravariant or natural gradient using the inverse of a Riemannian metric tensor, which is derived for the manifold of invertible matrix transfer functions. This has not been previously done using the language of differential geometry.

[1]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[2]  Todd K. Moon,et al.  Contravariant adaptation on structured matrix spaces , 2002, Signal Process..

[3]  S. Haykin.  Unsupervised adaptive filtering, vol. 1: Blind source separation , 2000 .

[4]  Steven Thomas Smith,et al.  Optimization Techniques on Riemannian Manifolds , 2014, ArXiv.

[5]  Hiroshi Sawada,et al.  Natural gradient multichannel blind deconvolution and speech separation using causal FIR filters , 2004, IEEE Transactions on Speech and Audio Processing.

[6]  Wojciech Czaja,et al.  Schroedinger Eigenmaps for the Analysis of Biomedical Data , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Chi-Tsong Chen,et al.  Linear System Theory and Design , 1995 .

[8]  S.C. Douglas,et al.  Multichannel blind deconvolution of arbitrary signals: adaptive algorithms and stability analyses , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[9]  S. Amari,et al.  Gradient Learning in Structured Parameter Spaces: Adaptive Blind Separation of Signal Sources , 1996 .

[10]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[11]  Shun-ichi Amari,et al.  Novel On-Line Adaptive Learning Algorithms for Blind Deconvolution Using the Natural Gradient Approach , 1997 .

[12]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[13]  Shun-ichi Amari,et al.  Adaptive blind signal processing-neural network approaches , 1998, Proc. IEEE.

[14]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[15]  Shun-ichi Amari,et al.  Why natural gradient? , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[16]  C. Udriste,et al.  Convex Functions and Optimization Methods on Riemannian Manifolds , 1994 .

[17]  M. T. Vaughn.  Geometry in Physics , 2008 .

[18]  Hrushikesh Narhar Mhaskar,et al.  Locally Learning Biomedical Data Using Diffusion Frames , 2012, J. Comput. Biol..