Bayesian Markov Switching Tensor Regression For Time-Varying Networks

We propose a new Bayesian Markov switching regression model for multi-dimensional arrays (tensors) of binary time series. We assume a zero-inflated logit dynamics with time-varying parameters and apply it to multi-layer temporal networks. The original contribution is threefold. First, in order to avoid over-fitting we propose a parsimonious parametrization of the model, based on a low-rank decomposition of the tensor of regression coefficients. Second, the parameters of the tensor model are driven by a hidden Markov chain, thus allowing for structural changes. The regimes are identified through prior constraints on the mixing probability of the zero-inflated model. Finally, we model the jointly dynamics of the network and of a set of variables of interest. We follow a Bayesian approach to inference, exploiting the Polya-Gamma data augmentation scheme for logit models in order to provide an efficient Gibbs sampler for posterior approximation. We show the effectiveness of the sampler on simulated datasets of medium-big sizes, finally we apply the methodology to a real dataset of financial networks.

[1]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[2]  Reinhard Diestel,et al.  Graph Theory , 1997 .

[3]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[4]  Sataya D. Dubey,et al.  Compound gamma, beta and F distributions , 1970 .

[5]  Jean-Marie Dufour,et al.  Testing Causality between Two Vectors in Multivariate Autoregressive Moving Average Models , 1992 .

[6]  Isabelle Mejean,et al.  Firms, Destinations, and Aggregate Fluctuations , 2014 .

[7]  Roberto Casarin,et al.  A Bayesian Markov-Switching Correlation Model for Contagion Analysis on Exchange Rate Markets , 2018 .

[8]  S. Frühwirth-Schnatter Markov chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models , 2001 .

[9]  W. Hackbusch Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[10]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[11]  Vasco M. Carvalho,et al.  The Network Origins of Aggregate Fluctuations , 2011 .

[12]  M. V. Visaya,et al.  Analysis of Binary Multivariate Longitudinal Data via 2-Dimensional Orbits: An Application to the Agincourt Health and Socio-Demographic Surveillance System in South Africa , 2015, PloS one.

[13]  James G. Scott,et al.  Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables , 2012, 1205.0310.

[14]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[15]  Z. Wang,et al.  The structure and dynamics of multilayer networks , 2014, Physics Reports.

[16]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[17]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[18]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[19]  D. Dunson,et al.  Bayesian network‐response regression , 2016, Bioinform..

[20]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[21]  Daniel F. Waggoner,et al.  Methods for Inference in Large Multiple-Equation Markov-Switching Models , 2006 .

[22]  Diane Lambert,et al.  Zero-inflacted Poisson regression, with an application to defects in manufacturing , 1992 .

[23]  D. V. van Dyk,et al.  Partially Collapsed Gibbs Samplers , 2008 .

[24]  B. Graham An Econometric Model of Network Formation With Degree Heterogeneity , 2017 .

[25]  Padhraic Smyth,et al.  Bayesian nonhomogeneous Markov models via Pólya-Gamma data augmentation with applications to rainfall modeling , 2017, 1701.02856.

[26]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[27]  Sylvia Kaufmann,et al.  Dating and forecasting turning points by Bayesian clustering with dynamic structure: A suggestion with an application to Austrian data , 2010 .

[28]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[29]  Mike West,et al.  Bayesian Forecasting of Many Count-Valued Time Series , 2018, Journal of Business & Economic Statistics.

[30]  Michael J. Dueker Markov Switching in GARCH Processes and Mean-Reverting Stock-Market Volatility , 1997 .

[31]  Michael Schweinberger,et al.  MAXIMUM LIKELIHOOD ESTIMATION FOR SOCIAL NETWORK DYNAMICS. , 2010, The annals of applied statistics.

[32]  Andrzej Cichocki,et al.  Fundamental Tensor Operations for Large-Scale Data Analysis in Tensor Train Formats , 2014, ArXiv.

[33]  N. Shephard,et al.  Markov chain Monte Carlo methods for stochastic volatility models , 2002 .

[34]  Fulvio Corsi,et al.  Realizing Smiles: Options Pricing with Realized Volatility , 2011 .

[35]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[36]  P. Kroonenberg Applied Multiway Data Analysis , 2008 .

[37]  Tatiyana V. Apanasovich,et al.  On estimation in binary autologistic spatial models , 2006 .

[38]  Peter D. Turney Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[39]  A. Zellner An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for Aggregation Bias , 1962 .

[40]  Jesse Windle,et al.  A Tractable State-Space Model for Symmetric Positive-Definite Matrices , 2013, 1310.5951.

[41]  R W Doerge,et al.  Variable Selection in High‐Dimensional Multivariate Binary Data with Application to the Analysis of Microbial Community DNA Fingerprints , 2002, Biometrics.

[42]  Gilles Celeux,et al.  Bayesian Inference for Mixture: The Label Switching Problem , 1998, COMPSTAT.

[43]  A. Lo,et al.  Econometric Measures of Connectedness and Systemic Risk in the Finance and Insurance Sectors , 2011 .

[44]  C. Sims,et al.  Were there Regime Switches in U.S. Monetary Policy , 2006 .

[45]  Norman R. Swanson,et al.  Impulse Response Functions Based on a Causal Approach to Residual Orthogonalization in Vector Autoregressions , 1997 .

[46]  M. Springer,et al.  The Distribution of Products of Beta, Gamma and Gaussian Random Variables , 1970 .

[47]  G. Kapetanios,et al.  Estimating the Dynamics and Persistence of Financial Networks, with an Application to the Sterling Money Market , 2016 .

[48]  N. Christakis,et al.  SUPPLEMENTARY ONLINE MATERIAL FOR: The Collective Dynamics of Smoking in a Large Social Network , 2022 .

[49]  Michael Wooldridge,et al.  Econometric Analysis of Cross Section and Panel Data, 2nd Edition , 2001 .

[50]  Daniele Durante,et al.  Bayesian Logistic Gaussian Process Models for Dynamic Networks , 2014, AISTATS.

[51]  Monica Billio,et al.  Modeling Systemic Risk with Markov Switching Graphical SUR Models , 2015, Journal of Econometrics.

[52]  M. West,et al.  High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics , 2008, Journal of the American Statistical Association.

[53]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[54]  Michael A. West,et al.  Dynamic matrix-variate graphical models , 2007 .

[55]  F. Diebold,et al.  UNIVERSITY OF SOUTHERN CALIFORNIA Center for Applied Financial Economics (CAFE) On the Network Topology of Variance Decompositions: Measuring the Connectedness of Financial Firms , 2011 .

[56]  D. F. Ahelegbey,et al.  Bayesian Graphical Models for Structural Vector Autoregressive Processes , 2012, Journal of Applied Econometrics.

[57]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[58]  James D. Hamilton,et al.  Autoregressive conditional heteroskedasticity and changes in regime , 1994 .

[59]  David B. Dunson,et al.  Bayesian Tensor Regression , 2015, J. Mach. Learn. Res..

[60]  P. Heagerty,et al.  Regression analysis of longitudinal binary data with time-dependent environmental covariates: bias and efficiency. , 2005, Biostatistics.

[61]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[62]  Martin A. Lindquist,et al.  Meta-analysis of functional neuroimaging data using Bayesian nonparametric binary regression , 2012, 1206.6674.

[63]  Daniel R. Smith Markov-Switching and Stochastic Volatility Diffusion Models of Short-Term Interest Rates , 2002 .

[64]  Giacomo Bormetti,et al.  Smile from the Past: A General Option Pricing Framework with Multiple Volatility and Leverage Components , 2014 .

[65]  Angelo Mele,et al.  A Structural Model of Dense Network Formation , 2017 .

[66]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[67]  Matt Taddy,et al.  Multinomial Inverse Regression for Text Analysis , 2010, 1012.2098.

[68]  Xi Chen,et al.  Scalable Bayesian Modeling, Monitoring, and Analysis of Dynamic Network Flow Data , 2016, 1607.02655.

[69]  Mark N. Harris,et al.  A zero-inflated ordered probit model, with an application to modelling tobacco consumption , 2007 .

[70]  James D. Hamilton A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle , 1989 .

[71]  Andrzej Cichocki,et al.  Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1 , 2016, ArXiv.

[72]  Giacomo Bormetti,et al.  Stochastic volatility with heterogeneous time scales , 2012, 1206.0026.

[73]  Klaassen Improving GARCH Volatility Forecasts with Regime-Switching GARCH Klaassen, F.J.G.M , 2001 .

[74]  D. F. Ahelegbey,et al.  Sparse Graphical Vector Autoregression: A Bayesian Approach , 2016 .

[75]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[76]  Marc S. Paolella,et al.  A New Approach to Markov-Switching GARCH Models , 2004 .

[77]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[78]  A. V. D. Vaart,et al.  Adaptive Bayesian density estimation with location-scale mixtures , 2010 .

[79]  Matt Taddy Autoregressive Mixture Models for Dynamic Spatial Poisson Processes: Application to Tracking Intensity of Violent Crime , 2010 .

[80]  Sylvia Kaufmann,et al.  Measuring business cycles with a dynamic Markov switching factor model : an assessment using Bayesian simulation methods , 1997 .

[81]  S. Kaufmann K-state switching models with time-varying transition distributions—Does loan growth signal stronger effects of variables on inflation? , 2015 .

[82]  Chang-Jin Kim,et al.  Business Cycle Turning Points, A New Coincident Index, and Tests of Duration Dependence Based on a Dynamic Factor Model With Regime Switching , 1998, Review of Economics and Statistics.

[83]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..