Geometricity and Embedding

In this chapter, we compare and contrast two approaches to the problem of embedding non-Euclidean data, namely geometric and structure preserving embedding. Under the first heading, we explore how spherical embedding can be used to embed data onto the surface of sphere of optimal radius. Here we explore both elliptic and hyperbolic geometries, i.e., positive and negative curvatures. Our results on synthetic and real data show that the elliptic embedding performs well under noisy conditions and can deliver low-distortion embeddings for a wide variety of datasets. Hyperbolic data seems to be much less common (at least in our datasets) and is more difficult to accurately embed. Under the second heading, we show how the Ihara zeta function can be used to embed hypergraphs in a manner which reflects their underlying relational structure. Specifically, we show how a polynomial characterization derived from the Ihara zeta function leads to an embedding which captures the prime cycle structure of the hypergraphs.

[1]  Horst Bunke,et al.  A graph distance metric based on the maximal common subgraph , 1998, Pattern Recognit. Lett..

[2]  Iwao Sato,et al.  Bartholdi zeta functions of graph coverings , 2003, J. Comb. Theory B.

[3]  Joachim M. Buhmann,et al.  Path-Based Clustering for Grouping of Smooth Curves and Texture Segmentation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Lawrence Hubert,et al.  Linear and circular unidimensional scaling for symmetric proximity matrices , 1997 .

[5]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[6]  Amnon Shashua,et al.  Linear image coding for regression and classification using the tensor-rank principle , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Serge J. Belongie,et al.  Higher order learning with graphs , 2006, ICML.

[8]  J. Rodríguez On the Laplacian Spectrum and Walk-regular Hypergraphs , 2003 .

[9]  L. Bartholdi Counting Paths in Graphs , 2000, math/0012161.

[10]  Amnon Shashua,et al.  A unifying approach to hard and probabilistic clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Edwin R. Hancock,et al.  Pattern Vectors from Algebraic Graph Theory , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Marianna Bolla,et al.  Spectra, Euclidean representations and clusterings of hypergraphs , 1993, Discret. Math..

[13]  Edwin R. Hancock,et al.  A Riemannian approach to graph embedding , 2007, Pattern Recognit..

[14]  Kenji Fukumizu,et al.  Graph Zeta Function in the Bethe Free Energy and Loopy Belief Propagation , 2009, NIPS.

[15]  Robert Krauthgamer,et al.  Metric Embeddings—Beyond One-Dimensional Distortion , 2004, Discret. Comput. Geom..

[16]  Kaspar Riesen,et al.  Approximate graph edit distance computation by means of bipartite graph matching , 2009, Image Vis. Comput..

[17]  Kevin Françoisse,et al.  The Sum-over-Paths Covariance Kernel: A Novel Covariance Measure between Nodes of a Directed Graph , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Dragoš Cvetković,et al.  EIGENVALUE BOUNDS FOR THE SIGNLESS LAPLACIAN , 2007 .

[19]  Geoffrey Scott,et al.  The coefficients of the Ihara zeta function , 2008 .

[20]  Andrea Torsello,et al.  A hypergraph-based approach to affine parameters estimation , 2008, ICPR.

[21]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Andrea Torsello,et al.  A game-theoretic approach to partial clique enumeration , 2009, Image Vis. Comput..

[23]  F. Chung The Laplacian of a Hypergraph. , 1992 .

[24]  Ki-ichiro Hashimoto,et al.  ARTIN TYPE L-FUNCTIONS AND THE DENSITY THEOREM FOR PRIME CYCLES ON FINITE GRAPHS , 1992 .

[25]  A. Terras,et al.  Zeta functions of finite graphs and coverings, III , 1996 .

[26]  Edwin R. Hancock,et al.  Learning shape-classes using a mixture of tree-unions , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[28]  M. Zaslavskiy,et al.  A Path Following Algorithm for the Graph Matching Problem , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Nikos Paragios,et al.  Graph commute times for image representation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[31]  Ernest Valveny,et al.  Generalized median graph computation by means of graph embedding in vector spaces , 2010, Pattern Recognit..

[32]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[33]  Joachim M. Buhmann,et al.  On the information and representation of non-Euclidean pairwise data , 2006, Pattern Recognit..

[34]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[35]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[36]  J. Rodri´guez On the Laplacian Eigenvalues and Metric Parameters of Hypergraphs , 2002 .

[37]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[38]  T. Caelli,et al.  Constant curvature Riemannian scaling , 1978 .

[39]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[40]  Shih-Fu Chang,et al.  Graph construction and b-matching for semi-supervised learning , 2009, ICML '09.

[41]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[42]  H. Bass THE IHARA-SELBERG ZETA FUNCTION OF A TREE LATTICE , 1992 .

[43]  Horst Bunke,et al.  Matching of Hypergraphs - Algorithms, Applications, and Experiments , 2008, Applied Pattern Recognition.

[44]  Edwin R. Hancock,et al.  Clustering and Embedding Using Commute Times , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Marc Rioux,et al.  Recognition and Shape Synthesis of 3-D Objects Based on Attributed Hypergraphs , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[47]  Jan Ramon,et al.  Expressivity versus efficiency of graph kernels , 2003 .

[48]  Xuelong Li,et al.  Non-negative graph embedding , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[50]  Edwin R. Hancock,et al.  Spectral Embedding of Feature Hypergraphs , 2008, SSPR/SPR.

[51]  K. Boyer,et al.  Organizing Large Structural Modelbases , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Yosi Keller,et al.  Efficient High Order Matching , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Deli Zhao,et al.  Cyclizing Clusters via Zeta Function of a Graph , 2008, NIPS.

[55]  P. Thomas Fletcher,et al.  Principal geodesic analysis for the study of nonlinear statistics of shape , 2004, IEEE Transactions on Medical Imaging.

[56]  Edwin R. Hancock,et al.  Spectral embedding of graphs , 2003, Pattern Recognit..

[57]  Edward Y. Chang,et al.  Learning with non-metric proximity matrices , 2005, MULTIMEDIA '05.

[58]  Tamir Hazan,et al.  Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization , 2006, ECCV.

[59]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[60]  Kaspar Riesen,et al.  Graph Classification by Means of Lipschitz Embedding , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[61]  Karsten M. Borgwardt,et al.  The skew spectrum of graphs , 2008, ICML '08.

[62]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[63]  Anil K. Jain,et al.  Representation and Recognition of Handwritten Digits Using Deformable Templates , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  C Cannings,et al.  Multi-player matrix games. , 1997, Bulletin of mathematical biology.

[65]  Jean Ponce,et al.  A Tensor-Based Algorithm for High-Order Graph Matching , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Horst Bunke,et al.  Non-Euclidean or Non-metric Measures Can Be Informative , 2006, SSPR/SPR.

[67]  Shuicheng Yan,et al.  Multiplicative nonnegative graph embedding , 2009, CVPR.

[68]  Amnon Shashua,et al.  Probabilistic graph and hypergraph matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Edwin R. Hancock,et al.  Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshop, SSPR&SPR 2010, Cesme, Izmir, Turkey, August 18-20, 2010. Proceedings , 2010, SSPR/SPR.

[70]  Ulrike von Luxburg,et al.  Influence of graph construction on graph-based clustering measures , 2008, NIPS.

[71]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[72]  Joachim M. Buhmann,et al.  Going Metric: Denoising Pairwise Data , 2002, NIPS.

[73]  Jiawei Han,et al.  Image clustering with tensor representation , 2005, ACM Multimedia.

[74]  King-Sun Fu,et al.  A distance measure between attributed relational graphs for pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[75]  Joachim M. Buhmann,et al.  Optimal Cluster Preserving Embedding of Nonmetric Proximity Data , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Edwin R. Hancock,et al.  Efficient computation of Ihara coefficients using the Bell polynomial recursion , 2012 .

[77]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[78]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[79]  Xavier L. Hubaut,et al.  Strongly regular graphs , 1975, Discret. Math..

[80]  Yuval Shavitt,et al.  Hyperbolic embedding of internet graph for distance estimation and overlay construction , 2008, TNET.

[81]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[82]  Pietro Perona,et al.  Beyond pairwise clustering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[83]  C. Blakemore,et al.  Analysis of connectivity in the cat cerebral cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[84]  Simone Severini,et al.  Coined quantum walks lift the cospectrality of graphs and trees , 2005, Pattern Recognit..

[85]  Marcello Pelillo,et al.  Dominant Sets and Pairwise Clustering , 2007 .

[86]  T. Sunada,et al.  Zeta Functions of Finite Graphs , 2000 .

[87]  Simone Severini,et al.  A Matrix Representation of Graphs and its Spectrum as a Graph Invariant , 2006, Electron. J. Comb..

[88]  Tony Jebara,et al.  Structure preserving embedding , 2009, ICML '09.

[89]  Edwin R. Hancock,et al.  Discovering Shape Classes using Tree Edit-Distance and Pairwise Clustering , 2007, International Journal of Computer Vision.

[90]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[91]  L. Beineke,et al.  Topics in algebraic graph theory , 2004 .

[92]  S. V. Savchenko The zeta-function and Gibbs measures , 1993 .

[93]  T. Cox,et al.  Multidimensional scaling on a sphere , 1991 .

[94]  Deng Cai,et al.  Tensor Subspace Analysis , 2005, NIPS.

[95]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[96]  Y. Ihara On discrete subgroups of the two by two projective linear group over p-adic fields , 1966 .

[97]  Edwin R. Hancock,et al.  Hypergraphs, Characteristic Polynomials and the Ihara Zeta Function , 2009, CAIP.

[98]  Daniel A. Spielman,et al.  Fitting a graph to vector data , 2009, ICML '09.

[99]  Guangliang Chen,et al.  Kernel Spectral Curvature Clustering (KSCC) , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[100]  Bernard P. Brooks,et al.  The coefficients of the characteristic polynomial in terms of the eigenvalues and the elements of an n×n matrix , 2006, Appl. Math. Lett..

[101]  Patrick Solé,et al.  Spectra of Regular Graphs and Hypergraphs and Orthogonal Polynomials , 1996, Eur. J. Comb..

[102]  Iwao Sato,et al.  A new Bartholdi zeta function of a graph , 2007 .

[103]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[104]  Haibin Ling,et al.  Shape Classification Using the Inner-Distance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[105]  Venu Madhav Govindu,et al.  A tensor decomposition for geometric grouping and segmentation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[106]  Dong Xu,et al.  Discriminant analysis with tensor representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[107]  Karsten M. Borgwardt,et al.  Fast subtree kernels on graphs , 2009, NIPS.

[108]  Ping Zhu,et al.  A study of graph spectra for comparing graphs and trees , 2008, Pattern Recognit..

[109]  Edwin R. Hancock,et al.  Graph characteristics from the heat kernel trace , 2009, Pattern Recognit..

[110]  King-Sun Fu,et al.  Subgraph error-correcting isomorphisms for syntactic pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[111]  Wan-Jui Lee,et al.  An Inexact Graph Comparison Approach in Joint Eigenspace , 2008, SSPR/SPR.

[112]  Christopher K. Storm The Zeta Function of a Hypergraph , 2006, Electron. J. Comb..

[113]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[114]  Marcello Pelillo,et al.  A Game-Theoretic Approach to Hypergraph Clustering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[115]  Jon M. Kleinberg,et al.  Clustering categorical data: an approach based on dynamical systems , 2000, The VLDB Journal.

[116]  Guangliang Chen,et al.  Foundations of a Multi-way Spectral Clustering Framework for Hybrid Linear Modeling , 2008, Found. Comput. Math..

[117]  R. Shankar,et al.  Principles of Quantum Mechanics , 2010 .

[118]  Lov K. Grover A fast quantum mechanical algorithm for database search , 1996, STOC '96.

[119]  Karsten M. Borgwardt,et al.  The graphlet spectrum , 2009, ICML '09.

[120]  Driss Aboutajdine,et al.  Hypergraph imaging: an overview , 2002, Pattern Recognit..

[121]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[122]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[123]  Edwin R. Hancock,et al.  A probabilistic spectral framework for grouping and segmentation , 2004, Pattern Recognit..

[124]  Gilad Lerman,et al.  On d-dimensional d-semimetrics and simplex-type inequalities for high-dimensional sine functions , 2008, J. Approx. Theory.

[125]  Hai Jin,et al.  Projective Nonnegative Graph Embedding , 2010, IEEE Transactions on Image Processing.