Solving Tensor Structured Problems with Computational Tensor Algebra

Since its introduction by Gauss, Matrix Algebra has facilitated understanding of scientific problems, hiding distracting details and finding more elegant and efficient ways of computational solving. Today's largest problems, which often originate from multidimensional data, might profit from even higher levels of abstraction. We developed a framework for solving tensor structured problems with tensor algebra that unifies concepts from tensor analysis, multilinear algebra and multidimensional signal processing. In contrast to the conventional matrix approach, it allows the formulation of multidimensional problems, in a multidimensional way, preserving structure and data coherence; and the implementation of automated optimizations of solving algorithms, based on the commutativity of all tensor operations. Its ability to handle large scientific tasks is showcased by a real-world, 4D medical imaging problem, with more than 30 million unknown parameters solved on a current, inexpensive hardware. This significantly surpassed the best published matrix-based approach.

[1]  Damien Muti,et al.  Multidimensional signal processing using lower-rank tensor approximation , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[2]  Michael Unser,et al.  Variational image reconstruction from arbitrarily spaced samples: a fast multiresolution spline solution , 2005, IEEE Transactions on Image Processing.

[3]  K. Raghavachari,et al.  Electron Correlation Techniques in Quantum Chemistry: Recent Advances , 1991 .

[4]  Mark S. Gordon,et al.  General atomic and molecular electronic structure system , 1993, J. Comput. Chem..

[5]  David E. Bernholdt,et al.  Synthesis of High-Performance Parallel Programs for a Class of ab Initio Quantum Chemistry Models , 2005, Proceedings of the IEEE.

[6]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[7]  Demetri Terzopoulos,et al.  Multilinear image analysis for facial recognition , 2002, Object recognition supported by user interaction for service robots.

[8]  A. Einstein The Foundation of the General Theory of Relativity , 1916 .

[9]  John R. Rice,et al.  Tensor product analysis of partial difference equations , 1964 .

[10]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[11]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[12]  A. Aldroubi,et al.  Wavelets in Medicine and Biology , 1997 .

[13]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[14]  Lars Eldén Multi-linear mappings, SVD, HOSVD, and the numerical solution of ill-conditioned tensor least squares problems , 2005 .

[15]  D. Leibovici,et al.  A singular value decomposition of a k-way array for a principal component analysis of multiway data, PTA-k , 1998 .

[16]  Ivan P. Gavrilyuk,et al.  Hierarchical Tensor-Product Approximation to the Inverse and Related Operators for High-Dimensional Elliptic Problems , 2004, Computing.

[17]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[18]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[19]  Pierre Comon,et al.  Tensor Decompositions, State of the Art and Applications , 2002 .

[20]  Richard E. Blahut,et al.  Fast Algorithms for Digital Signal Processing , 1985 .

[21]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[22]  N. Ahuja,et al.  Compact representation of multidimensional data using tensor rank-one decomposition , 2004, ICPR 2004.

[23]  Yonghong Zeng,et al.  Transforms and Fast Algorithms for Signal Analysis and Representations , 2003 .

[24]  Bülent Yener,et al.  Unsupervised Multiway Data Analysis: A Literature Survey , 2009, IEEE Transactions on Knowledge and Data Engineering.

[25]  Y. Saad,et al.  Iterative solution of linear systems in the 20th century , 2000 .

[26]  L. Lathauwer,et al.  From Matrix to Tensor : Multilinear Algebra and Signal Processing , 1996 .

[27]  J. Heinbockel Introduction to Tensor Calculus and Continuum Mechanics , 2001 .

[28]  V. Pereyra,et al.  Efficient Computer Manipulation of Tensor Products with Applications to Multidimensional Approximation , 1973 .