Supervised Dimension Reduction by Local Neighborhood Optimization for Image Processing

Subspace learning-based dimensionality reduction algorithms are important and have been popularly applied in data mining, pattern recognition and computer vision applications. They show the successful dimension reduction when data points are evenly distributed in the high-dimensional space. However, some may distort the local geometric structure of the original dataset and result in a poor low-dimensional embedding while data samples show an uneven distribution in the original space. In this paper, we propose a supervised dimension reduction method by local neighborhood optimization to disposal the uneven distribution of high-dimensional data. It extends the widely used Locally Linear Embedding (LLE) framework, namely LNOLLE. The method considers the class label of the data to optimize local neighborhood, which achieves better separability inter-class distance of the data in the low-dimensional space with the aim to abstain holding together the data samples of different classes while mapping an uneven distributed data. This effectively preserves the geometric topological structure of the original data points. We use the presented LNOLLE method to the image classification and face recognition, which achieves a good classification result and higher face recognition accuracy compared with existing manifold learning methods including popular supervised algorithms. In addition, we consider the reconstruction of the method to solve noise suppression for seismic image. To the best of our knowledge, this is the first manifold learning approach to solve high-dimensional nonlinear seismic data for noise suppression. The experimental results on forward model and real seismic data show that LNOLLE improves signal to noise ratio of seismic image compared with the widely used Singular Value Decomposition (SVD) filtering method.

[1]  Abdallah Bashir Musa A comparison of ℓ1-regularizion, PCA, KPCA and ICA for dimensionality reduction in logistic regression , 2013, International Journal of Machine Learning and Cybernetics.

[2]  Michael I. Jordan,et al.  Sufficient dimension reduction for visual sequence classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Germán Castellanos-Domínguez,et al.  Locally linear embedding based on correntropy measure for visualization and classification , 2012, Neurocomputing.

[4]  Michael I. Jordan,et al.  Dimensionality Reduction for Spectral Clustering , 2011, AISTATS.

[5]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[6]  Heng Huang,et al.  Large Margin Local Estimate With Applications to Medical Image Classification , 2015, IEEE Transactions on Medical Imaging.

[7]  Jin Young Choi,et al.  Linear boundary discriminant analysis , 2010, Pattern Recognit..

[8]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[9]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[10]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[11]  Ying Wah Teh,et al.  Text mining of news-headlines for FOREX market prediction: A Multi-layer Dimension Reduction Algorithm with semantics and sentiment , 2015, Expert Syst. Appl..

[12]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Guanglu Sun,et al.  Active Learning Method for Chinese Spam Filtering , 2017 .

[14]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[15]  Amir Akramin Shafie,et al.  Robust face recognition against expressions and partial occlusions , 2016, Int. J. Autom. Comput..

[16]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[17]  Michael I. Jordan,et al.  Unsupervised Kernel Dimension Reduction , 2010, NIPS.

[18]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[19]  Enrico Magli,et al.  Compressed Fingerprint Matching and Camera Identification via Random Projections , 2015, IEEE Transactions on Information Forensics and Security.

[20]  A. Posadas,et al.  Spatial‐temporal analysis of a seismic series using the principal components method: The Antequera Series, Spain, 1989 , 1993 .

[21]  Bing Li,et al.  Principal support vector machines for linear and nonlinear sufficient dimension reduction , 2011, 1203.2790.

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  Danasingh Asir Antony Gnana Singh,et al.  An unsupervised feature selection algorithm with feature ranking for maximizing performance of the classifiers , 2015, Int. J. Autom. Comput..

[24]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[25]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[26]  Anazida Zainal,et al.  An adaptive and efficient dimension reduction model for multivariate wireless sensor networks applications , 2013, Appl. Soft Comput..

[27]  Yehuda Koren,et al.  Ieee Transactions on Visualization and Computer Graphics Robust Linear Dimensionality Reduction , 2022 .

[28]  Matemáticas Nonlinear Dimensionality Reduction , 2013 .

[29]  Peter J. Bickel,et al.  Maximum Likelihood Estimation of Intrinsic Dimension , 2004, NIPS.

[30]  Tony Jebara,et al.  Structure preserving embedding , 2009, ICML '09.