Feature Fusion via Tensor Network Summation

Tensor networks (TNs) have been earning considerable attention as multiway data analysis tools owing to their ability to tackle the curse of dimensionality through the representation of large-scale tensors via smaller-scale interconnections of their intrinsic features. However, despite the obvious benefits, the current treatment of TNs as stand-alone entities does not take full advantage of their underlying structure and the associated feature localization. To this end, we exploit the analogy with feature fusion to propose a rigorous framework for the combination of TNs, with a particular focus on their summation as a natural way of their combination. The proposed framework is shown to allow for feature combination of any number of tensors, as long as their TN representation topologies are isomorphic. Simulations involving multi-class classification of an image dataset show the benefits of the proposed framework.

[1]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[2]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[3]  R. Bro PARAFAC. Tutorial and applications , 1997 .

[4]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[5]  A. Constantinides,et al.  A unified method for segmentation and edge detection using graph theory , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[7]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[8]  Andrzej Cichocki,et al.  Common and Individual Feature Extraction Using Tensor Decompositions: a Remedy for the Curse of Dimensionality? , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Masashi Sugiyama,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives , 2017, Found. Trends Mach. Learn..

[10]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[11]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[12]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[13]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Andrzej Cichocki,et al.  Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction , 2012, IEEE Transactions on Neural Networks and Learning Systems.