This study proposes a novel dimensionality reduction (DR) method for multi-view datasets. The principal component analysis (PCA) idea of minimising least squares reconstruction errors is extended to consider both data distribution and penalty weights called dictionary to recover outliers free global structures from missing and noisy data points. In this way, PCA is viewed as a special instance of the authors' proposed dictionary induced least squares framework (DLS). Furthermore, to appropriately handle multi-view DR, we combine the DLS with multiple manifold embeddings (DLSME). Therefore it can obtain lower projections while maintaining a balance between preserving global structures with DLS and local structures with multi-manifold embeddings. Extensive experiments on object and face recognition datasets verify that the DLS achieves better classification results with lower dimensional projections than PCA. Also, on many multi-view datasets of visual recognition and web image annotation, the DLSME method demonstrates more effectiveness than Graph-Laplacian PCA (gLPCA), robust PCA-optimal mean, canonical correlation analysis (CCA), bilinear models (BLM), neighbourhood preserving embedding, locality preserving projections, and locality sensitive discriminant analysis.