Linear to multi-linear algebra and systems using tensors

In past few decades, tensor algebra also known as multi-linear algebra has been developed and customized as a tool to be used for various engineering applications. In particular, with the help of a special form of tensor contracted product, known as the Einstein Product and its properties, many of the known concepts from Linear Algebra could be extended to a multi-linear setting. This enables to define the notions of multi-linear system theory where the input, output signals and the system are multi-domain in nature. This paper provides an overview of tensor algebra tools which can be seen as an extension of linear algebra, at the same time highlighting the difference and advantages that the multi-linear setting brings forth. In particular, the notion of tensor inversion, tensor singular value and tensor Eigenvalue decomposition using the Einstein product is explained. In addition, this paper also introduces the notion of contracted convolution in both discrete and continuous multi-linear system tensors. Tensor Networks representation of various tensor operations is also presented. Also, application of tensor tools in developing transceiver schemes for multi-domain communication systems, with an example of MIMO CDMA systems, is presented. Thus this paper acts as an entry point tutorial for graduate students whose research involves multi-domain or multi-modal signals and systems.

[1]  Harry Leib,et al.  Multi-Domain Communication Systems and Networks: A Tensor-Based Approach , 2021, Network.

[2]  H. Leib,et al.  A Tensor based Precoder and Receiver for MIMO GFDM systems , 2021, ICC 2021 - IEEE International Conference on Communications.

[3]  Hongyang Chen,et al.  Tensor Decompositions in Wireless Communications and MIMO Radar , 2021, IEEE Journal of Selected Topics in Signal Processing.

[4]  Baohua Huang,et al.  Numerical study on Moore-Penrose inverse of tensors via Einstein product , 2021, Numerical Algorithms.

[5]  Amit Surana,et al.  Multilinear Control Systems Theory , 2020, SIAM J. Control. Optim..

[6]  Baohua Huang,et al.  Numerical subspace algorithms for solving the tensor equations involving Einstein product , 2020, Numer. Linear Algebra Appl..

[7]  Qingwen Wang,et al.  Iterative algorithms for solving some tensor equations , 2019 .

[8]  L. Qi,et al.  Bernstein Concentration Inequalities for Tensors via Einstein Products , 2019, 1902.03056.

[9]  Amit Surana,et al.  Multilinear Time Invariant System Theory , 2019, 2019 Proceedings of the Conference on Control and its Applications.

[10]  Changfeng Ma,et al.  An iterative algorithm to solve the generalized Sylvester tensor equations , 2018, Linear and Multilinear Algebra.

[11]  B. Zheng,et al.  Tensor inversion and its application to the tensor equations with Einstein product , 2018, Linear and Multilinear Algebra.

[12]  Krushnachandra Panigrahy,et al.  Extension of Moore–Penrose inverse of tensor via Einstein product , 2018, Linear and Multilinear Algebra.

[13]  Anru R. Zhang,et al.  Tensor SVD: Statistical and Computational Limits , 2017, IEEE Transactions on Information Theory.

[14]  Nikos D. Sidiropoulos,et al.  Tensors for Data Mining and Data Fusion , 2016, ACM Trans. Intell. Syst. Technol..

[15]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[16]  Michael K. Ng,et al.  An eigenvalue problem for even order tensors with its applications , 2016 .

[17]  Changjiang Bu,et al.  Moore–Penrose inverse of tensors via Einstein product , 2016 .

[18]  Ran Pan,et al.  Tensor Transpose and Its Properties , 2014, ArXiv.

[19]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[20]  Yunming Ye,et al.  MultiComm: Finding Community Structure in Multi-Dimensional Networks , 2014, IEEE Transactions on Knowledge and Data Engineering.

[21]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[22]  Andrzej Cichocki,et al.  Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions , 2014, ArXiv.

[23]  Na Li,et al.  Solving Multilinear Systems via Tensor Inversion , 2013, SIAM J. Matrix Anal. Appl..

[24]  Chen Ling,et al.  On determinants and eigenvalue theory of tensors , 2013, J. Symb. Comput..

[25]  Andrzej Cichocki,et al.  Multiway array decomposition analysis of EEGs in Alzheimer's disease , 2012, Journal of Neuroscience Methods.

[26]  Siep Weiland,et al.  Singular Value Decompositions and Low Rank Approximations of Tensors , 2010, IEEE Transactions on Signal Processing.

[27]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[28]  Ana Cristina Murillo,et al.  Localization and Matching Using the Planar Trifocal Tensor With Bearing-Only Data , 2008, IEEE Transactions on Robotics.

[29]  Toshiyuki Tanaka,et al.  Asymptotic Analysis of General Multiuser Detectors in MIMO DS-CDMA Channels , 2007, IEEE Journal on Selected Areas in Communications.

[30]  Lieven De Lathauwer,et al.  Fourth-Order Cumulant-Based Blind Identification of Underdetermined Mixtures , 2007, IEEE Transactions on Signal Processing.

[31]  R. Bro.  Review on Multiway Analysis in Chemistry—2000–2005 , 2006 .

[32]  Tamara G. Kolda,et al.  Categories and Subject Descriptors: G.4 [Mathematics of Computing]: Mathematical Software— , 2022 .

[33]  Giorgio Taricco,et al.  Linear receivers for the multiple-input multiple-output multiple-access channel , 2006, IEEE Transactions on Communications.

[34]  Lek-Heng Lim,et al.  Singular values and eigenvalues of tensors: a variational approach , 2005, 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005..

[35]  Liqun Qi,et al.  Eigenvalues of a real supersymmetric tensor , 2005, J. Symb. Comput..

[36]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[37]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[38]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[39]  E. Davidson,et al.  Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents , 1981 .

[40]  H. Leib,et al.  The Tensor Multi-linear Channel and its Shannon Capacity , 2022, IEEE Access.

[41]  Harry Leib,et al.  Tensor Multi-linear MMSE Estimation Using the Einstein Product , 2021 .

[42]  Harry Leib,et al.  A Tensor Framework for Multi-Linear Complex MMSE Estimation , 2021, IEEE Open Journal of Signal Processing.

[43]  Harry Leib,et al.  A Tensor Based Framework for Multi-Domain Communication Systems , 2020, IEEE Open Journal of the Communications Society.

[44]  D. Mandic,et al.  Tensor Decompositions and Practical Applications: A Hands-on Tutorial , 2019, INNSBDDL.

[45]  Sabrina Hirsch,et al.  Digital Signal Processing A Computer Based Approach , 2016 .

[46]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[47]  aw Kula.  Fundamentals of Communication Systems , 2022 .