A weighted view on the partial least-squares algorithm

In this paper it is shown that the Partial Least-Squares (PLS) algorithm for univariate data is equivalent to using a truncated Cayley-Hamilton polynomial expression of degree [email protected][email protected]?r for the matrix inverse (X^TX)^-^[email protected]?R^r^x^r which is used to compute the least-squares (LS) solution. Furthermore, the a coefficients in this polynomial are computed as the optimal LS solution (minimizing parameters) to the prediction error. The resulting solution is non-iterative. The solution can be expressed in terms of a matrix inverse and is given by B"P"L"S=K"a(K"a^TX^TXK"a)^-^1K"a^TX^TY where K"[email protected]?R^r^x^a is the controllability (Krylov) matrix for the pair (X^TX,X^TY). The iterative PLS algorithm for computing the orthogonal weighting matrix W"a as presented in the literature, is shown here to be equivalent to computing an orthonormal basis (using, e.g. the QR algorithm) for the column space of K"a. The PLS solution can equivalently be computed as B"P"L"S=W"a(W"a^TX^TXW"a)^-^1W"a^TX^TY, where W"a is the Q (orthogonal) matrix from the QR decomposition K"a=W"aR. Furthermore, we have presented an optimal and non-iterative truncated Cayley-Hamilton polynomial LS solution for multivariate data. The free parameters in this solution is found as the minimizing solution of a prediction error criterion.