Concentration for matrix martingales in continuous time and microscopic activity of social networks

This paper gives new concentration inequalities for the spectral norm of matrix martingales in continuous time. Both cases of purely discountinuous and continuous martingales are considered. The analysis is based on a new supermartingale property of the trace exponential, based on tools from stochastic calculus. Matrix martingales in continuous time are probabilistic objects that naturally appear for statistical learning of time-dependent systems. We focus here on the the microscopic study of (social) networks, based on self-exciting counting processes, such as the Hawkes process, together with a low-rank prior assumption of the self-exciting component. A consequence of these new concentration inequalities is a push forward of the theoretical analysis of such models.

[1]  P. Brémaud.  Point processes and queues, martingale dynamics , 1983 .

[2]  M. Wegkamp,et al.  Optimal selection of reduced rank estimators of high-dimensional matrices , 2010, 1004.2995.

[3]  E. Bacry,et al.  Linear processes in high dimensions: Phase space and critical properties. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Alex Gittens,et al.  The spectral norm error of the naive Nystrom extension , 2011, ArXiv.

[5]  K. Markström,et al.  Expansion properties of random Cayley graphs and vertex transitive graphs via matrix martingales , 2008 .

[6]  P. Bartlett,et al.  Empirical minimization , 2006 .

[7]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[8]  Le Song,et al.  Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes , 2013, AISTATS.

[9]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[10]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Theory of martingales , 1989 .

[11]  A. Tsybakov,et al.  Estimation of high-dimensional low-rank matrices , 2009, 0912.5338.

[12]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[13]  Faperj.  Sums of random Hermitian matrices and an inequality by Rudelson , 2010 .

[14]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[15]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[16]  Agathe Guilloux,et al.  High-dimensional additive hazards models and the Lasso , 2011, 1106.4662.

[17]  Padhraic Smyth,et al.  Stochastic blockmodeling of relational event dynamics , 2013, AISTATS.

[18]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[19]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[20]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[21]  Rudolf Ahlswede,et al.  Strong converse for identification via quantum channels , 2000, IEEE Trans. Inf. Theory.

[22]  R. Oliveira.  Concentration of the adjacency matrix and of the Laplacian in random graphs with independent edges , 2009, 0911.0600.

[23]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[24]  Shuang-Hong Yang,et al.  Mixture of Mutually Exciting Processes for Viral Diffusion , 2013, ICML.

[25]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[26]  A. Shiryaev,et al.  Limit Theorems for Stochastic Processes , 1987 .

[27]  P. Bremaud,et al.  Point Processes and Queues: Martingale Dynamics (Springer Series in Statistics) , 1981 .

[28]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[29]  R. Latala.  Some estimates of norms of random matrices , 2005 .

[30]  Stanislav Minsker.  On Some Extensions of Bernstein's Inequality for Self-adjoint Operators , 2011, 1112.5448.

[31]  A. Hawkes.  Spectra of some self-exciting and mutually exciting point processes , 1971 .

[32]  Yoav Seginer,et al.  The Expected Norm of Random Matrices , 2000, Combinatorics, Probability and Computing.

[33]  Didier Sornette,et al.  Robust dynamic classes revealed by measuring the response function of a social system , 2008, Proceedings of the National Academy of Sciences.

[34]  Lasso and probabilistic inequalities for multivariate point processes , 2015, 1208.0570.

[35]  Tomoharu Iwata,et al.  Discovering latent influence in online social activities via shared cascade poisson processes , 2013, KDD.

[36]  Ameet Talwalkar,et al.  Divide-and-Conquer Matrix Factorization , 2011, NIPS.

[37]  Katherine A. Heller,et al.  Modelling Reciprocating Relationships with Hawkes Processes , 2012, NIPS.

[38]  Michael I. Jordan,et al.  Matrix concentration inequalities via the method of exchangeable pairs , 2012, 1201.6002.

[39]  Scott W. Linderman,et al.  Discovering Latent Network Structure in Point Process Data , 2014, ICML.

[40]  R. Bhatia.  Matrix Analysis , 1996 .

[41]  E. Lieb.  Convex trace functions and the Wigner-Yanase-Dyson conjecture , 1973 .

[42]  S. Geer.  Exponential Inequalities for Martingales, with Application to Maximum Likelihood Estimation for Counting Processes , 1995 .

[43]  Bernhard Schölkopf,et al.  Modeling Information Propagation with Survival Theory , 2013, ICML.

[44]  V. Koltchinskii,et al.  Oracle inequalities in empirical risk minimization and sparse recovery problems , 2011 .

[45]  Le Song,et al.  Estimating Diffusion Network Structures: Recovery Conditions, Sample Complexity & Soft-thresholding Algorithm , 2014, ICML.

[46]  Bernhard Schölkopf,et al.  Uncovering the Temporal Dynamics of Diffusion Networks , 2011, ICML.

[47]  P. Reynaud-Bouret.  Compensator and exponential inequalities for some suprema of counting processes , 2006 .

[48]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[49]  D. Petz.  A survey of certain trace inequalities , 1994 .

[50]  J. Tropp.  FREEDMAN'S INEQUALITY FOR MATRIX MARTINGALES , 2011, 1101.3039.

[51]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.