Online Graph-Adaptive Learning With Scalability and Privacy

Graphs are widely adopted for modeling complex systems, including financial, biological, and social networks. Nodes in networks usually entail attributes, such as the age or gender of users in a social network. However, real-world networks can have very large size, and nodal attributes can be unavailable to a number of nodes, e.g., due to privacy concerns. Moreover, new nodes can emerge over time, which can necessitate real-time evaluation of their nodal attributes. In this context, this paper deals with scalable learning of nodal attributes by estimating a nodal function based on noisy observations at a subset of nodes. A multikernel-based approach is developed, which is scalable to large-size networks. Unlike most existing methods that re-solve the function estimation problem over all existing nodes whenever a new node joins the network, the novel method is capable of providing real-time evaluation of the function values on newly joining nodes without resorting to a batch solver. Interestingly, the novel scheme only relies on an encrypted version of each node's connectivity in order to learn the nodal attributes, which promotes privacy. Experiments on both synthetic and real datasets corroborate the effectiveness of the proposed methods.

[1]  Sergio Barbarossa,et al.  Adaptive Graph Signal Processing: Algorithms and Optimal Sampling Strategies , 2017, IEEE Transactions on Signal Processing.

[2]  Mehryar Mohri,et al.  On Transductive Regression , 2006, NIPS.

[3]  Georgios B. Giannakis,et al.  Kernel-Based Reconstruction of Graph Signals , 2016, IEEE Transactions on Signal Processing.

[4]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[5]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[6]  Yousef Saad,et al.  Fast Approximate kNN Graph Construction for High Dimensional Data via Recursive Lanczos Bisection , 2009, J. Mach. Learn. Res..

[7]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[8]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[9]  Santiago Segarra,et al.  Sampling of Graph Signals With Successive Local Aggregations , 2015, IEEE Transactions on Signal Processing.

[10]  Sunil K. Narang,et al.  Signal processing techniques for interpolation in graph structured data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Pengfei Liu,et al.  Local-Set-Based Graph Signal Reconstruction , 2014, IEEE Transactions on Signal Processing.

[12]  N. Altman An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[13]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[14]  Steven C. H. Hoi,et al.  Large Scale Online Kernel Learning , 2016, J. Mach. Learn. Res..

[15]  Georgios B. Giannakis,et al.  Online Ensemble Multi-kernel Learning Adaptive to Non-stationary and Adversarial Environments , 2017, AISTATS.

[16]  Risi Kondor,et al.  Diffusion kernels on graphs and other discrete structures , 2002, ICML 2002.

[17]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[18]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[19]  Yizhou Sun,et al.  Mining heterogeneous information networks: a structural analysis approach , 2013, SKDD.

[20]  Georgios B. Giannakis,et al.  Topology inference of multilayer networks , 2017, 2017 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[21]  Mason A. Porter,et al.  Multilayer networks , 2013, J. Complex Networks.

[22]  L. Getoor,et al.  Link-Based Classification , 2003, Encyclopedia of Machine Learning and Data Mining.

[23]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[24]  Georgios B. Giannakis,et al.  Topology Identification and Learning over Graphs: Accounting for Nonlinearities and Dynamics , 2018, Proceedings of the IEEE.

[25]  Hongyuan Zha,et al.  Co-ranking Authors and Documents in a Heterogeneous Network , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[26]  Georgios B. Giannakis,et al.  Semi-Blind Inference of Topologies and Dynamical Processes over Graphs , 2018, ArXiv.

[27]  Pascal Frossard,et al.  Learning Laplacian Matrix in Smooth Graph Signal Representations , 2014, IEEE Transactions on Signal Processing.

[28]  Georgios B. Giannakis,et al.  Inference of Spatio-Temporal Functions Over Graphs via Multikernel Kriged Kalman Filtering , 2017, IEEE Transactions on Signal Processing.

[29]  Larry A. Wasserman,et al.  Statistical Analysis of Semi-Supervised Regression , 2007, NIPS.

[30]  Jason Weston,et al.  Transductive Inference for Estimating Values of Functions , 1999, NIPS.

[31]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[32]  Georgios B. Giannakis,et al.  Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics , 2017, J. Mach. Learn. Res..

[33]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[34]  Georgios B. Giannakis,et al.  Adaptive Diffusions for Scalable Learning Over Graphs , 2018, IEEE Transactions on Signal Processing.

[35]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[36]  G. Wahba Spline models for observational data , 1990 .

[37]  Eric D. Kolaczyk,et al.  Statistical Analysis of Network Data: Methods and Models , 2009 .

[38]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[39]  Charles A. Micchelli,et al.  Learning the Kernel Function via Regularization , 2005, J. Mach. Learn. Res..