Learning Heat Diffusion Graphs

Information analysis of data often boils down to properly identifying their hidden structure. In many cases, the data structure can be described by a graph representation that supports signals in the dataset. In some applications, this graph may be partly determined by design constraints or predetermined sensing arrangements. In general though, the data structure is not readily available nor easily defined. In this paper, we propose to represent structured data as a sparse combination of localized functions that live on a graph. This model is more appropriate to represent local data arrangements than the classical global smoothness prior. We focus on the problem of inferring the connectivity that best explains the data samples at different vertices of a graph that is a priori unknown. We concentrate on the case where the observed data are actually the sum of heat diffusion processes, which is a widely used model for data on networks or other irregular structures. We cast a new graph learning problem and solve it with an efficient nonconvex optimization algorithm. Experiments on both synthetic and real world data finally illustrate the benefits of the proposed graph learning framework and confirm that the data structure can be efficiently learned from data observations only. We believe that our algorithm will help solving key questions in diverse application domains such as social and biological network analysis where it is crucial to unveil proper data structure for understanding and inference.

[1]  Samuel D. Relton,et al.  A Block Krylov Method to Compute the Action of the Fréchet Derivative of a Matrix Function on a Vector with Applications to Condition Number Estimation , 2017, SIAM J. Sci. Comput..

[2]  Fan Chung,et al.  The heat kernel as the pagerank of a graph , 2007, Proceedings of the National Academy of Sciences.

[3]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[4]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[5]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[6]  Pascal Frossard,et al.  Learning Laplacian Matrix in Smooth Graph Signal Representations , 2014, IEEE Transactions on Signal Processing.

[7]  Rémi Gribonval,et al.  Should Penalized Least Squares Regression be Interpreted as Maximum A Posteriori Estimation? , 2011, IEEE Transactions on Signal Processing.

[8]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[9]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[10]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[11]  B. Bollobás The evolution of random graphs , 1984 .

[12]  Antonio Ortega,et al.  Generalized Laplacian precision matrix estimation for graph signal processing , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[14]  Pascal Frossard,et al.  Learning Parametric Dictionaries for Signals on Graphs , 2014, IEEE Transactions on Signal Processing.

[15]  Vassilis Kalofolias,et al.  How to Learn a Graph from Smooth Signals , 2016, AISTATS.

[16]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[17]  F. Girardi,et al.  The field campaigns of the European Tracer Experiment (ETEX): overview and results , 1998 .

[18]  Mikhail Belkin,et al.  Towards a Theoretical Foundation for Laplacian-Based Manifold Methods , 2005, COLT.

[19]  Xavier Bresson,et al.  Source localization on graphs via ℓ1 recovery and spectral graph theory , 2016, 2016 IEEE 12th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP).

[20]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[21]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[22]  Michael R. Lyu,et al.  Mining social networks using heat diffusion processes for marketing candidates selection , 2008, CIKM '08.

[23]  A. Rbnyi ON THE EVOLUTION OF RANDOM GRAPHS , 2001 .

[24]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[25]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[26]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[27]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[28]  José M. F. Moura,et al.  Signal processing on graphs: Estimating the structure of a graph , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  W. Marsden I and J , 2012 .

[30]  Michael G. Rabbat,et al.  Characterization and Inference of Graph Diffusion Processes From Observations of Stationary Signals , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[31]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[32]  Santiago Segarra,et al.  Network Topology Inference from Spectral Templates , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[33]  Pascal Frossard,et al.  Learning of structured graph dictionaries , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Joshua B. Tenenbaum,et al.  Discovering Structure by Learning Sparse Graphs , 2010 .

[35]  David J. Bartholomew,et al.  Latent Variable Models and Factor Analysis: A Unified Approach , 2011 .

[36]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[37]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[38]  Vincent Gripon,et al.  Characterization and inference of weighted graph topologies from observations of diffused signals , 2016, ArXiv.

[39]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.