Stepwise estimation of common principal components

The standard common principal components (CPCs) may not always be useful for simultaneous dimensionality reduction in k groups. Moreover, the original FG algorithm finds the CPCs in arbitrary order, which does not reflect their importance with respect to the explained variance. A possible alternative is to find an approximate common subspace for all k groups. A new stepwise estimation procedure for obtaining CPCs is proposed, which imitates standard PCA. The stepwise CPCs facilitate simultaneous dimensionality reduction, as their variances are decreasing at least approximately in all k groups. Thus, they can be a better alternative for dimensionality reduction than the standard CPCs. The stepwise CPCs are found sequentially by a very simple algorithm, based on the well-known power method for a single covariance/correlation matrix. Numerical illustrations on well-known data are considered.