An Information-Theoretic Approach to Self-Organisation: Emergence of Complex Interdependencies in Coupled Dynamical Systems

Self-organisation lies at the core of fundamental but still unresolved scientific questions, and holds the promise of de-centralised paradigms crucial for future technological developments. While self-organising processes have been traditionally explained by the tendency of dynamical systems to evolve towards specific configurations, or attractors, we see self-organisation as a consequence of the interdependencies that those attractors induce. Building on this intuition, in this work we develop a theoretical framework for understanding and quantifying self-organisation based on coupled dynamical systems and multivariate information theory. We propose a metric of global structural strength that identifies when self-organisation appears, and a multi-layered decomposition that explains the emergent structure in terms of redundant and synergistic interdependencies. We illustrate our framework on elementary cellular automata, showing how it can detect and characterise the emergence of complex structures.

[1]  M. Esposito,et al.  Three faces of the second law. I. Master equation formulation. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  Luc Steels,et al.  Synthesising the origins of language and meaning using co-evolution, self-organisation and level formation , 1998 .

[3]  Bart de Boer,et al.  Self-organization in vowel systems , 2000, J. Phonetics.

[4]  Yu. L. Klimontovich,et al.  Turbulent Motion. The Structure of Chaos , 1991 .

[5]  H. Schuster Deterministic chaos: An introduction , 1984 .

[6]  S. Lloyd,et al.  Measures of complexity: a nonexhaustive list , 2001 .

[7]  Kwang-Cheng Chen,et al.  A Technological Perspective on Information Cascades via Social Learning , 2017, IEEE Access.

[8]  H. Haken,et al.  Synergetics , 1988, IEEE Circuits and Devices Magazine.

[9]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[10]  Lai-Sang Young,et al.  What Are SRB Measures, and Which Dynamical Systems Have Them? , 2002 .

[11]  S. Wolfram Random sequence generation by cellular automata , 1986 .

[12]  Steffen Staab,et al.  Neurons, Viscose Fluids, Freshwater Polyp Hydra-and Self-Organizing Information Systems , 2003, IEEE Intell. Syst..

[13]  Henrik Jeldtoft Jensen,et al.  Statistical mechanics of exploding phase spaces: ontic open systems , 2016, Journal of Physics A: Mathematical and Theoretical.

[14]  S. Kak Information, physics, and computation , 1996 .

[15]  Adam B. Barrett,et al.  An exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Carlos Gershenson,et al.  The Meaning of Self-organization in Computing , 2003 .

[17]  Christian Bettstetter,et al.  Self-organization in communication networks: principles and design paradigms , 2005, IEEE Communications Magazine.

[18]  Paul M. B. Vitányi,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 1993, Graduate Texts in Computer Science.

[19]  Falko Dressler,et al.  A study of self-organization mechanisms in ad hoc and sensor networks , 2008, Comput. Commun..

[20]  Paola Flocchini,et al.  Solving the parity problem in one-dimensional cellular automata , 2013, Natural Computing.

[21]  Francis Heylighen,et al.  The Science of Self-Organization and Adaptivity , 1999 .

[22]  Tânia Tomé,et al.  Entropy production in nonequilibrium systems at stationary states. , 2012, Physical review letters.

[23]  Yaneer Bar-Yam,et al.  Multiscale Complexity/Entropy , 2004, Adv. Complex Syst..

[24]  J. Kelso,et al.  Self-organization of coordinative movement patterns ☆ , 1988 .

[25]  Mikhail Prokopenko,et al.  Guided Self-Organization: Inception , 2014 .

[26]  Jürgen Kurths,et al.  Synchronization - A Universal Concept in Nonlinear Sciences , 2001, Cambridge Nonlinear Science Series.

[27]  Marian Verhelst,et al.  Understanding Interdependency Through Complex Information Sharing , 2015, Entropy.

[28]  Lawrence S. Schulman,et al.  Time''s arrow and quantum measurement , 1997 .

[29]  James P. Crutchfield,et al.  dit: a Python package for discrete information theory , 2018, J. Open Source Softw..

[30]  W. Ebeling Stochastic Processes in Physics and Chemistry , 1995 .

[31]  R. Xu,et al.  Theory of open quantum systems , 2002 .

[32]  Thomas M. Cover,et al.  Network Information Theory , 2001 .

[33]  Bulcsú Sándor,et al.  The Sensorimotor Loop as a Dynamical System: How Regular Motion Primitives May Emerge from Self-Organized Limit Cycles , 2015, Front. Robot. AI.

[34]  Joseph T. Lizier,et al.  Information Decomposition of Target Effects from Multi-Source Interactions: Perspectives on Previous, Current and Future Work , 2018, Entropy.

[35]  Georgi Georgiev,et al.  Self-organization in non-equilibrium systems , 2015 .

[36]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[37]  Carlos Gershenson,et al.  Complexity and information: Measuring emergence, self-organization, and homeostasis at multiple scales , 2012, Complex..

[38]  H. Von Foerster,et al.  Principles of Self-Organization: Transactions of the University of Illinois Symposium , 1962 .

[39]  J. Crutchfield,et al.  Measures of statistical complexity: Why? , 1998 .

[40]  W. Ashby,et al.  Principles of the self-organizing dynamic system. , 1947, The Journal of general psychology.

[41]  Joseph D. Bryngelson,et al.  Thermodynamics of chaotic systems: An introduction , 1994 .

[42]  H. Haken Synergetics: an Introduction, Nonequilibrium Phase Transitions and Self-organization in Physics, Chemistry, and Biology , 1977 .

[43]  P. N. Kugler,et al.  On the concept of coordinative structures as dissipative structures: i , 1980 .

[44]  Nazim Fatès,et al.  A Guided Tour of Asynchronous Cellular Automata , 2013, J. Cell. Autom..

[45]  Albert Y. Zomaya,et al.  The local information dynamics of distributed computation in complex systems , 2012 .

[46]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[47]  Robin A. A. Ince The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal , 2017, ArXiv.

[48]  Hector Zenil,et al.  Computation and Universality: Class IV versus Class III Cellular Automata , 2013, J. Cell. Autom..

[49]  Stephen Wolfram,et al.  A New Kind of Science , 2003, Artificial Life.

[50]  R. Robinson,et al.  An Introduction to Dynamical Systems: Continuous and Discrete , 2004 .

[51]  Robin A. A. Ince Measuring multivariate redundant information with pointwise common change in surprisal , 2016, Entropy.

[52]  I. Prigogine,et al.  Formative Processes. (Book Reviews: Self-Organization in Nonequilibrium Systems. From Dissipative Structures to Order through Fluctuations) , 1977 .

[53]  Joseph T. Lizier,et al.  Pointwise Partial Information DecompositionUsing the Specificity and Ambiguity Lattices , 2018, Entropy.

[54]  D. Chalmers Strong and Weak Emergence , 2006 .

[55]  Eckehard Olbrich,et al.  Information Decomposition and Synergy , 2015, Entropy.

[56]  Marian Verhelst,et al.  Understanding high-order correlations using a synergy-based decomposition of the total entropy , 2015 .

[57]  Darcy W. E. Allen,et al.  Blockchains and the Boundaries of Self-Organized Economies: Predictions for the Future of Banking , 2016 .

[58]  Stephen Wolfram,et al.  Universality and complexity in cellular automata , 1983 .

[59]  S G Nurzaman,et al.  Goal-directed multimodal locomotion through coupling between mechanical and attractor selection dynamics. , 2015, Bioinspiration & biomimetics.

[60]  Melanie Mitchell,et al.  Computation in Cellular Automata: A Selected Review , 2005, Non-standard Computation.

[61]  Patrick Tague,et al.  Network self-organization in the Internet of Things , 2013, 2013 IEEE International Conference on Sensing, Communications and Networking (SECON).

[62]  D. Chemla,et al.  Lives of the artists. , 2008, Age and ageing.

[63]  E. Ott Chaos in Dynamical Systems: Contents , 1993 .

[64]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[65]  Te Sun Han Nonnegative Entropy Measures of Multivariate Symmetric Correlations , 1978, Inf. Control..

[66]  M. Crommelinck,et al.  Self-organization and emergence in life sciences , 2006 .

[67]  Gianpiero Cattaneo,et al.  Investigating topological chaos by elementary cellular automata dynamics , 2000, Theor. Comput. Sci..

[68]  Yongsheng Ding,et al.  An Intelligent Self-Organization Scheme for the Internet of Things , 2013, IEEE Computational Intelligence Magazine.

[69]  Carlos Gershenson,et al.  When Can We Call a System Self-Organizing? , 2003, ECAL.

[70]  P. N. Kugler,et al.  1 On the Concept of Coordinative Structures as Dissipative Structures: I. Theoretical Lines of Convergence* , 1980 .

[71]  U. Seifert Stochastic thermodynamics, fluctuation theorems and molecular machines , 2012, Reports on progress in physics. Physical Society.

[72]  Eckehard Olbrich,et al.  The information theory of individuality , 2014, Theory in Biosciences.

[73]  James P. Crutchfield,et al.  Anatomy of a Spin: The Information-Theoretic Structure of Classical Spin Systems , 2017, Entropy.

[74]  Carlos Gershenson,et al.  Guiding the self-organization of random Boolean networks , 2010, Theory in Biosciences.

[75]  Francesco Petruccione,et al.  The Theory of Open Quantum Systems , 2002 .

[76]  G. Edelman,et al.  A measure for brain complexity: relating functional segregation and integration in the nervous system. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[77]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[78]  Randall D. Beer,et al.  Nonnegative Decomposition of Multivariate Information , 2010, ArXiv.

[79]  Robert Haslinger,et al.  Quantifying self-organization with optimal predictors. , 2004, Physical review letters.

[80]  Yoshiki Kuramoto,et al.  Chemical Oscillations, Waves, and Turbulence , 1984, Springer Series in Synergetics.

[81]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[82]  E. Simoncini,et al.  Self-organization in dissipative structures: A thermodynamic theory for the emergence of prebiotic cells and their epigenetic evolution , 2009, Biosyst..

[83]  Yaneer Bar-Yam,et al.  Multiscale Information Theory and the Marginal Utility of Information , 2017, Entropy.

[84]  Masayuki Murata,et al.  Controlling Large-Scale Self-Organized Networks with Lightweight Cost for Fast Adaptation to Changing Environments , 2016, TAAS.

[85]  J. Kelso,et al.  The Metastable Brain , 2014, Neuron.

[86]  Karl J. Friston,et al.  Characterising the complexity of neuronal interactions , 1995 .

[87]  Gavan Lintern,et al.  Dynamic patterns: The self-organization of brain and behavior , 1997, Complex.

[88]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[89]  Marco Tomassini,et al.  Cryptography with cellular automata , 2001, Appl. Soft Comput..

[90]  Holger Kantz,et al.  Noise in chaotic data: Diagnosis and treatment. , 1995, Chaos.

[91]  H. Von Foerster,et al.  On Self-Organizing Systems and Their Environments , 2003 .

[92]  W. Ross Ashby,et al.  Principles of the Self-Organizing System , 1991 .

[93]  Gregory J. Chaitin,et al.  Information, Randomness & Incompleteness - Papers on Algorithmic Information Theory - Second Edition , 1997 .

[94]  V. Isaeva Self-organization in biological systems , 2012, Biology Bulletin.

[95]  D. Lathrop Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering , 2015 .

[96]  Dirk Oliver Theis,et al.  BROJA-2PID: A Robust Estimator for Bivariate Partial Information Decomposition , 2018, Entropy.

[97]  Franco Zambonelli,et al.  Case studies for self-organization in computer science , 2006, J. Syst. Archit..