Information processing features can detect behavioral regimes of dynamical systems

In dynamical systems, local interactions between dynamical units generate correlations which are stored and transmitted throughout the system, generating the macroscopic behavior. However a framework to quantify and study this at the microscopic scale is missing. Here we propose an 'information processing' framework based on Shannon mutual information quantities between the initial and future states. We apply it to the 256 elementary cellular automata (ECA), which are the simplest possible dynamical systems exhibiting behaviors ranging from simple to complex. Our main finding for ECA is that only a few features are needed for full predictability and that the 'information integration' (synergy) feature is always most predictive. Finally we apply the formalism to foreign exchange (FX) and interest-rate swap (IRS) time series data and find that the 2008 financial crisis marks a sudden and sustained regime shift (FX and EUR IRS) resembling tipping point behavior. The USD IRS market exhibits instead a slow and steady progression which appears consistent with the hypothesis that this market is (part of) the driving force behind the crisis. Our work suggests that the proposed framework is a promising way of predicting emergent complex systemic behaviors in terms of the local information processing of units.

[1]  Klaus Sutner,et al.  Computational classification of cellular automata , 2012, Int. J. Gen. Syst..

[2]  Bastien Chopard,et al.  Assessing complexity by means of maximum entropy models , 2014 .

[3]  Karel Culik,et al.  Undecidability of CA Classification Schemes , 1988, Complex Syst..

[4]  Benjamin Flecker,et al.  Multivariate information measures: an experimentalist's perspective , 2011, 1111.6857.

[5]  Anthony Saunders,et al.  Credit Risk Management In and Out of the Financial Crisis: New Approaches to Value at Risk and Other Paradigms , 2010 .

[6]  P. Grassberger.  Toward a quantitative theory of self-generated complexity , 1986 .

[7]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[8]  Eckehard Olbrich,et al.  Information Decomposition and Synergy , 2015, Entropy.

[9]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[10]  M. Scheffer,et al.  Complexity theory and financial regulation , 2016, Science.

[11]  Tomas Havranek,et al.  Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators , 2014 .

[12]  D. Kandhai,et al.  Efficient Computation of Exposure Profiles for Counterparty Credit Risk , 2014 .

[13]  Mark P. Taylor,et al.  The Crisis in the Foreign Exchange Market , 2009, SSRN Electronic Journal.

[14]  Peter M. A. Sloot,et al.  Quantifying Synergistic Information Using Intermediate Stochastic Variables , 2016, Entropy.

[15]  Randall D. Beer,et al.  Information Flow through a Model of the C. elegans Klinotaxis Circuit , 2015, PloS one.

[16]  Marten Scheffer,et al.  Spatial correlation as leading indicator of catastrophic shifts , 2010, Theoretical Ecology.

[17]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[18]  Randall D. Beer,et al.  Information Processing and Dynamics in Minimally Cognitive Agents , 2015, Cogn. Sci..

[19]  Joseph T. Lizier,et al.  Towards a synergy-based approach to measuring information modification , 2013, 2013 IEEE Symposium on Artificial Life (ALife).

[20]  Albert Y. Zomaya,et al.  The local information dynamics of distributed computation in complex systems , 2012 .

[21]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  P. Latham,et al.  Synergy, Redundancy, and Independence in Population Codes, Revisited , 2005, The Journal of Neuroscience.

[23]  Randall D. Beer,et al.  Nonnegative Decomposition of Multivariate Information , 2010, ArXiv.

[24]  Yaneer Bar-Yam,et al.  An Information-Theoretic Formalism for Multiscale Structure in Complex Systems , 2014, 1409.4708.

[25]  D. Garlaschelli,et al.  Early-warning signals of topological collapse in interbank networks , 2013, Scientific Reports.

[26]  Tracey Ho,et al.  Quantifying Redundant Information in Predicting a Target Random Variable , 2014, Entropy.

[27]  Michael J. Berry,et al.  Synergy, Redundancy, and Independence in Population Codes , 2003, The Journal of Neuroscience.

[28]  Tommy W. S. Chow,et al.  Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information , 2005, IEEE Transactions on Neural Networks.

[29]  Yaneer Bar-Yam,et al.  Computationally tractable pairwise complexity profile , 2012, Complex..

[30]  Christopher G. Langton,et al.  Computation at the edge of chaos: Phase transitions and emergent computation , 1990 .

[31]  James P. Crutchfield,et al.  Intersection Information Based on Common Randomness , 2013, Entropy.

[32]  Albert Y. Zomaya,et al.  Information modification and particle collisions in distributed computation. , 2010, Chaos.

[33]  Albert Y. Zomaya,et al.  The Information Dynamics of Phase Transitions in Random Boolean Networks , 2008, ALIFE.

[34]  Kristian Lindgren,et al.  An Information-Theoretic Perspective on Coarse-Graining, Including the Transition from Micro to Macro , 2015, Entropy.

[35]  Stephen Wolfram,et al.  A New Kind of Science , 2003, Artificial Life.

[36]  James P. Crutchfield,et al.  Anatomy of a Bit: Information in a Time Series Observation , 2011, Chaos.

[37]  Marten Scheffer,et al.  Slowing down as an early warning signal for abrupt climate change , 2008, Proceedings of the National Academy of Sciences of the United States of America.

[38]  S. Carpenter,et al.  Early-warning signals for critical transitions , 2009, Nature.

[39]  Rick Quax,et al.  The diminishing role of hubs in dynamical processes on complex networks , 2011, Journal of The Royal Society Interface.

[40]  Drona Kandhai,et al.  Information dissipation as an early-warning signal for the Lehman Brothers collapse in financial time series , 2013, Scientific Reports.