A class of multidimensional NIPALS algorithms for quaternion and tensor partial least squares regression

Abstract Quaternion and tensor-based signal processing benefits from exploiting higher dimensional structure in data to outperform the corresponding approaches using multivariate real algebras. Along with the extended range of processing options, these methods produce opportunities for a physically-meaningful interpretation. In this paper, we propose a class of novel partial least squares (PLS) algorithms for tensor- and quaternion-valued data, the widely linear quaternion PLS (WL-QPLS), the higher order nonlinear iterative PLS (HONIPALS) and the generalised higher order PLS (GHOPLS). This enables a regularised regression solution along with a latent variable decomposition of the original data based on the mutual information in the input and output block. The performance of the proposed algorithms is verified through analysis, together with a detailed comparison between quaternions and tensors and their application for image classification.

[1]  Danilo P. Mandic,et al.  A quaternion frequency estimator for three-phase power systems , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Biagio Palumbo,et al.  Analysis of profiles for monitoring of modern ship performance via partial least squares methods , 2018, Qual. Reliab. Eng. Int..

[3]  Jacob Benesty,et al.  Study of the quaternion LMS and four-channel LMS algorithms , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[5]  Kazuyuki Aihara,et al.  Quaternion-valued short term joint forecasting of three-dimensional wind and atmospheric parameters , 2011 .

[6]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[8]  R. Bro.  Multiway calibration. Multilinear PLS , 1996 .

[9]  Lilong Shi,et al.  Quaternion color texture segmentation , 2007, Comput. Vis. Image Underst..

[10]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Danilo P. Mandic,et al.  Augmented second-order statistics of quaternion random signals , 2011, Signal Process..

[12]  Danilo P. Mandic,et al.  An online NIPALS algorithm for Partial Least Squares , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Andrzej Cichocki,et al.  Common and Individual Feature Extraction Using Tensor Decompositions: a Remedy for the Curse of Dimensionality? , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Vladimir Risojevic,et al.  Unsupervised Quaternion Feature Learning for Remote Sensing Image Classification , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  H. Abdi.  Partial least squares regression and projection on latent structure regression (PLS Regression) , 2010 .

[16]  Francesco Grigoli,et al.  Optimal reorientation of geophysical sensors: A quaternion-based analytical solution , 2015 .

[17]  Danilo P. Mandic,et al.  Cost-effective quaternion minimum mean square error estimation: From widely linear to four-channel processing , 2017, Signal Process..

[18]  Danilo P. Mandic,et al.  A Distributed Quaternion Kalman Filter With Applications to Smart Grid and Target Tracking , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[19]  Danilo P. Mandic,et al.  Real-time estimation of quaternion impropriety , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[20]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[21]  Vladimir Risojevic,et al.  Unsupervised learning of quaternion features for image classification , 2013, 2013 11th International Conference on Telecommunications in Modern Satellite, Cable and Broadcasting Services (TELSIKS).

[22]  Danilo P. Mandic,et al.  Simultaneous diagonalisation of the covariance and complementary covariance matrices in quaternion widely linear signal processing , 2017, Signal Process..

[23]  Ignacio Santamaría,et al.  Properness and Widely Linear Processing of Quaternion Random Vectors , 2010, IEEE Transactions on Information Theory.

[24]  Masashi Sugiyama,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives , 2017, Found. Trends Mach. Learn..

[25]  T. Kolda.  Multilinear operators for higher-order decompositions , 2006 .

[26]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[27]  Esa Ollila,et al.  On the Circularity of a Complex Random Variable , 2008, IEEE Signal Processing Letters.

[28]  Danilo P. Mandic,et al.  Widely linear complex partial least squares for latent subspace regression , 2018, Signal Process..

[29]  Roman Rosipal,et al.  Overview and Recent Advances in Partial Least Squares , 2005, SLSFS.

[30]  Nicolas Le Bihan,et al.  Singular value decomposition of quaternion matrices: a new tool for vector-sensor signal processing , 2004, Signal Process..

[31]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Nicolas Le Bihan,et al.  Quaternion principal component analysis of color images , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[33]  S. Wold,et al.  PLS-regression: a basic tool of chemometrics , 2001 .

[34]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[35]  S. Wold,et al.  The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses , 1984 .