Optimality criteria for principal component analysis and generalizations

Principal components analysis can be derived from various criteria. Because these give essentially the same results, the question of which criterion should be used has not received much attention. Nevertheless, it can be argued that the approach of Pearson and Eckart & Young, based on variance explained by components, is more elegant and flexible than the (more popular) approach of Hotelling, which is concerned with variance that components have rather than explain. When two or more correlation or covariance matrices, based on the same variables, are to be analyzed in generalized component analysis, the question of which criterion is used becomes of utmost importance. A taxonomy of generalized principal component methods is given. It appears that generalized component analysis based on the Hotelling criterion coincides with one particular generalization based on the criterion of Pearson and Eckart & Young.