Research and implementation of SVD in machine learning

With the arrival of the era of big data, people's ability to collect and obtain data is becoming more powerful. These data have shown the characteristics of high dimension, large scale and complex structure. High dimensional data has seriously hindered the efficiency of data mining algorithm, we call it "the Dimension disaster ". Therefore, dimension reduction technology has become the primary task of big data mining and machine learning. In this paper, we focus on the method of data reduction, described the category of data dimension reduction. The research status and main algorithms of dimension reduction method are described in detail. This paper briefly introduces the latest research progress of data dimension reduction algorithm, including some popular algorithm such as PCA, KPCA, SVD, etc. The principle of principal component analysis (PCA) is discussed in this article, and the singular value decomposition (SVD) theorem is introduced to solve the problem that the PCA method has a large amount of computation, we also give a comparison of PCA and SVD. Finally, we design and implement some experiments to verify the application of SVD in data analysis and latent semantic indexing.

[1]  Yong-Seok Kim,et al.  Face Tracking and Recognition in Video with PCA-based Pose-Classification and (2D) 2 PCA recognition algorithm , 2013 .

[2]  Jin Hyun Park,et al.  Process monitoring using a Gaussian mixture model via principal component analysis and discriminant analysis , 2004, Comput. Chem. Eng..

[3]  Dean P. Foster,et al.  New Subsampling Algorithms for Fast Least Squares Regression , 2013, NIPS.

[4]  Jianhua Z. Huang,et al.  Sparse principal component analysis via regularized low rank matrix approximation , 2008 .

[5]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[6]  G. M. Jackson,et al.  Principal component transforms of triaxial recordings by singular value decomposition , 1991 .

[7]  Daoqiang Zhang,et al.  (2D)2PCA: Two-directional two-dimensional PCA for efficient face representation and recognition , 2005, Neurocomputing.

[8]  Qiuyong Zhao,et al.  Combination of Improved PCA and LDA for Video-based Face Recognition ⋆ , 2012 .

[9]  Turgay Çelik,et al.  Unsupervised Change Detection in Satellite Images Using Principal Component Analysis and $k$-Means Clustering , 2009, IEEE Geoscience and Remote Sensing Letters.

[10]  Yili Hong,et al.  Reliability Meets Big Data: Opportunities and Challenges , 2014 .

[11]  Meikang Qiu,et al.  Performance and Power Analysis of High-Density Multi-GPGPU Architectures: A Preliminary Case Study , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[12]  Tad J. Ulrych,et al.  Application of singular value decomposition to vertical seismic profiling , 1988 .

[13]  Jin Chen,et al.  Short-time matrix series based singular value decomposition for rolling bearing fault diagnosis , 2013 .