Signed Graph Metric Learning via Gershgorin Disc Perfect Alignment

Given a convex and differentiable objective [Formula: see text] for a real symmetric matrix [Formula: see text] in the positive definite (PD) cone, we propose a fast general metric learning framework that is entirely projection-free. We first assume that [Formula: see text] resides in a space [Formula: see text] of generalized graph Laplacian matrices corresponding to balanced signed graphs. Unlike low-rank metric matrices common in the literature, [Formula: see text] includes the important diagonal-only matrices as a special case. The key theorem to circumvent full eigen-decomposition and enable fast metric matrix optimization is Gershgorin disc perfect alignment (GDPA): given [Formula: see text] and diagonal matrix §, where Sii = 1/vi and is the first eigenvector of [Formula: see text], we prove that Gershgorin disc left-ends of similarity transform [Formula: see text] are perfectly aligned at the smallest eigenvalue λmin. Using this theorem, we replace the PD cone constraint with tightest possible linear constraints per iteration, so that the alternating optimization of the diagonal / off-diagonal terms in [Formula: see text] can be solved efficiently as linear programs via the Frank-Wolfe method. We update using Locally Optimal Block Preconditioned Conjugate Gradient with warm start as entries in [Formula: see text] are optimized successively. Experiments show that our graph metric optimization is significantly faster than cone-projection schemes, and produces competitive binary classification performance.

[1]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[2]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[3]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[4]  Wei Hu,et al.  Feature Graph Learning for 3D Point Cloud Denoising , 2019, IEEE Transactions on Signal Processing.

[5]  M. R. Rao,et al.  Combinatorial Optimization , 1992, NATO ASI Series.

[6]  Yang Song,et al.  Learning Fine-Grained Image Similarity with Deep Ranking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Yadong Mu,et al.  Fixed-Rank Supervised Metric Learning on Riemannian Manifold , 2016, AAAI.

[8]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[9]  Ljubisa Stankovic,et al.  Introduction to Graph Signal Processing , 2018, Signals and Communication Technology.

[10]  Xiang Zhang,et al.  Metric Learning from Relative Comparisons by Minimizing Squared Residual , 2012, 2012 IEEE 12th International Conference on Data Mining.

[11]  Yuanchao Bai,et al.  Fast Graph Sampling Set Selection Using Gershgorin Disc Alignment , 2019, IEEE Transactions on Signal Processing.

[12]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Tat-Seng Chua,et al.  An efficient sparse metric learning in high-dimensional space via l1-penalized log-determinant regularization , 2009, ICML '09.

[14]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Gene Cheung,et al.  Graph Laplacian Regularization for Image Denoising: Analysis in the Continuous Domain , 2016, IEEE Transactions on Image Processing.

[16]  Gert R. G. Lanckriet,et al.  Robust Structural Metric Learning , 2013, ICML.

[17]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[18]  Peter F. Stadler,et al.  Nodal Domain Theorems and Bipartite Subgraphs , 2005 .

[19]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face Verification in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Xiaochen Yang,et al.  Learning Local Metrics and Influential Regions for Classification , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Gene Cheung,et al.  Alternating Binary Classifier and Graph Learning from Partial Labels , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[22]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[23]  Jiwen Lu,et al.  Deep Metric Learning for Visual Understanding: An Overview of Recent Advances , 2017, IEEE Signal Processing Magazine.

[24]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[25]  Wei Hu,et al.  Graph Metric Learning via Gershgorin Disc Alignment , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Thorsten Joachims,et al.  Learning a Distance Metric from Relative Comparisons , 2003, NIPS.

[27]  Jure Leskovec,et al.  Signed networks in social media , 2010, CHI.

[28]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[29]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[30]  Enrico Magli,et al.  Graph Spectral Image Processing , 2018, Proceedings of the IEEE.

[31]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[32]  Rongrong Ji,et al.  Low-Rank Similarity Metric Learning in High Dimensions , 2015, AAAI.

[33]  Harvey Mudd,et al.  Irreducible Graphs , 2005 .

[34]  Silvio Savarese,et al.  Deep Metric Learning via Lifted Structured Feature Embedding , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[36]  Andrew V. Knyazev,et al.  Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method , 2001, SIAM J. Sci. Comput..

[37]  Lorenzo Torresani,et al.  Large Margin Component Analysis , 2006, NIPS.

[38]  F. Harary,et al.  STRUCTURAL BALANCE: A GENERALIZATION OF HEIDER'S THEORY1 , 1977 .

[39]  Yuanchao Bai,et al.  Reconstruction-cognizant Graph Sampling Using Gershgorin Disc Alignment , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  R. Varga.  Geršgorin And His Circles , 2004 .

[41]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[42]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[43]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Gene Cheung,et al.  Graph fourier transform with negative edges for depth image coding , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[45]  Suvrit Sra,et al.  Geometric Mean Metric Learning , 2016, ICML.

[46]  Lijun Zhang,et al.  Efficient Stochastic Optimization for Low-Rank Distance Metric Learning , 2017, AAAI.