Spectral Simplicity of Apparent Complexity, Part I: The Nondiagonalizable Metadynamics of Prediction

Virtually all questions that one can ask about the behavioral and structural complexity of a stochastic process reduce to a linear algebraic framing of a time evolution governed by an appropriate hidden-Markov process generator. Each type of question-correlation, predictability, predictive cost, observer synchronization, and the like-induces a distinct generator class. Answers are then functions of the class-appropriate transition dynamic. Unfortunately, these dynamics are generically nonnormal, nondiagonalizable, singular, and so on. Tractably analyzing these dynamics relies on adapting the recently introduced meromorphic functional calculus, which specifies the spectral decomposition of functions of nondiagonalizable linear operators, even when the function poles and zeros coincide with the operator's spectrum. Along the way, we establish special properties of the spectral projection operators that demonstrate how they capture the organization of subprocesses within a complex system. Circumventing the spurious infinities of alternative calculi, this leads in the sequel, Part II [P. M. Riechers and J. P. Crutchfield, Chaos 28, 033116 (2018)], to the first closed-form expressions for complexity measures, couched either in terms of the Drazin inverse (negative-one power of a singular operator) or the eigenvalues and projection operators of the appropriate transition dynamic.

[1]  James P. Crutchfield,et al.  Nearly Maximally Predictive Features and Their Dimensions , 2017, Physical review. E.

[2]  Robert Shaw,et al.  The Dripping Faucet As A Model Chaotic System , 1984 .

[3]  N. Dunford Spectral theory. I. Convergence to projections , 1943 .

[4]  Daniel Ray Upper,et al.  Theory and algorithms for hidden Markov models and generalized hidden Markov models , 1998 .

[5]  J. Crutchfield The calculi of emergence: computation, dynamics and induction , 1994 .

[6]  D. Cvetkovic,et al.  Spectra of Graphs: Theory and Applications , 1997 .

[7]  James P. Crutchfield,et al.  Structure and Randomness of Continuous-Time, Discrete-Event Processes , 2017, ArXiv.

[8]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[9]  V. Balasubramanian Equivalence and Reduction of Hidden Markov Models , 1993 .

[10]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[11]  James P. Crutchfield,et al.  Computational Mechanics: Pattern and Prediction, Structure and Simplicity , 1999, ArXiv.

[12]  Melville S. Green,et al.  Markoff Random Processes and the Statistical Mechanics of Time‐Dependent Phenomena. II. Irreversible Processes in Fluids , 1954 .

[13]  James P. Crutchfield,et al.  Asymptotic Synchronization for Finite-State Sources , 2010, ArXiv.

[14]  Hidden Markov models for stochastic thermodynamics , 2015 .

[15]  James P. Crutchfield,et al.  Predictive Rate-Distortion for Infinite-Order Markov Processes , 2016 .

[16]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[17]  Susanne Still,et al.  Optimal causal inference: estimating stored information and approximating causal architecture. , 2007, Chaos.

[18]  James P Crutchfield,et al.  Time's barbed arrow: irreversibility, crypticity, and stored information. , 2009, Physical review letters.

[19]  James P. Crutchfield,et al.  Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computation , 2010, Chaos.

[20]  Eckehard Olbrich,et al.  Comparison between Different Methods of Level Identification , 2014, Adv. Complex Syst..

[21]  James P. Crutchfield,et al.  Quantum automata and quantum grammars , 2000, Theor. Comput. Sci..

[22]  Paul Adrien Maurice Dirac,et al.  A new notation for quantum mechanics , 1939, Mathematical Proceedings of the Cambridge Philosophical Society.

[23]  R. Brualdi Spectra of digraphs , 2010 .

[24]  U. Seifert Stochastic thermodynamics, fluctuation theorems and molecular machines , 2012, Reports on progress in physics. Physical Society.

[25]  O. Penrose Foundations of Statistical Mechanics: A Deductive Treatment , 2005 .

[26]  C. Papadimitriou,et al.  Introduction to the Theory of Computation , 2018 .

[27]  James P. Crutchfield,et al.  Occam’s Quantum Strop: Synchronizing and Compressing Classical Cryptic Processes via a Quantum Channel , 2015, Scientific Reports.

[28]  James P. Crutchfield,et al.  Leveraging Environmental Correlations: The Thermodynamics of Requisite Variety , 2016, ArXiv.

[29]  P. Gaspard,et al.  Noise, chaos, and (ε,τ)-entropy per unit time , 1993 .

[30]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[31]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[32]  A. U.S.,et al.  Predictability , Complexity , and Learning , 2002 .

[33]  Almut Beige,et al.  Hidden Quantum Markov Models and Open Quantum Systems with Instantaneous Feedback , 2014, 1406.5847.

[34]  James P. Crutchfield,et al.  Exact Synchronization for Finite-State Sources , 2010, ArXiv.

[36]  Spectral Theory , 2017 .

[37]  Nicholas F. Travers Exponential Bounds for Convergence of Entropy Rate Approximations in Hidden Markov Models Satisfying a Path-Mergeability Condition , 2012, 1211.6181.

[38]  P. Grassberger Toward a quantitative theory of self-generated complexity , 1986 .

[39]  James P. Crutchfield,et al.  The Elusive Present: Hidden Past and Future Dependency and Why We Build Models , 2015, Physical review. E.

[40]  E Marseglia,et al.  An Introduction to X-Ray Crystallography , 1979 .

[41]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[42]  Eitan M. Gurari,et al.  Introduction to the theory of computation , 1989 .

[43]  James P. Crutchfield,et al.  Prediction, Retrodiction, and the Amount of Information Stored in the Present , 2009, ArXiv.

[44]  Franziska Wulf,et al.  Mathematical Population Genetics , 2016 .

[45]  A. Fisher,et al.  The Theory of critical phenomena , 1992 .

[46]  B. Nordstrom FINITE MARKOV CHAINS , 2005 .

[47]  Paul Davies,et al.  The Many Faces of State Space Compression , 2017 .

[48]  James Odell,et al.  Between order and chaos , 2011, Nature Physics.

[49]  M. Rahe,et al.  Finitary codings and weak Bernoulli partitions , 1979 .

[50]  James P. Crutchfield,et al.  How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence , 2011, Chaos.

[51]  James P. Crutchfield,et al.  Informational and Causal Architecture of Continuous-time Renewal Processes , 2016, 1611.01099.

[52]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[53]  A. Motter,et al.  Synchronization is optimal in nondiagonalizable networks. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[54]  Bart De Moor,et al.  Equivalence of state representations for hidden Markov models , 2007, 2007 European Control Conference (ECC).

[55]  Petre Stoica,et al.  Spectral Analysis of Signals , 2009 .

[56]  James P. Crutchfield,et al.  Anatomy of a Bit: Information in a Time Series Observation , 2011, Chaos.

[57]  R. Zwanzig,et al.  Time-Correlation Functions and Transport Coefficients in Statistical Mechanics , 1965 .

[58]  Rainer Klages,et al.  Nonequilibrium statistical physics of small systems : fluctuation relations and beyond , 2013 .

[59]  James P. Crutchfield,et al.  Time resolution dependence of information measures for spiking neurons: scaling and universality , 2015, Front. Comput. Neurosci..

[60]  M. Nowak Evolutionary Dynamics: Exploring the Equations of Life , 2006 .

[61]  Michael M Woolfson,et al.  An Introduction to X-ray Crystallography by Michael M. Woolfson , 1997 .

[62]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[63]  James P. Crutchfield,et al.  Exact Complexity: The Spectral Decomposition of Intrinsic Computation , 2013, ArXiv.

[64]  James P. Crutchfield,et al.  Information accessibility and cryptic processes , 2009, 0905.4787.

[65]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[66]  Naftali Tishby,et al.  Past-future information bottleneck in dynamical systems. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[67]  N. Packard,et al.  Symbolic dynamics of noisy chaos , 1983 .

[68]  Nihat Ay,et al.  Non-sufficient Memories That Are Sufficient for Prediction , 2009, Complex.

[69]  John J. Birch Approximations for the Entropy for Functions of Markov Chains , 1962 .

[70]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[71]  L. Goddard Information Theory , 1962, Nature.

[72]  James P. Crutchfield,et al.  Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse? , 2018, Chaos.

[73]  James P. Crutchfield,et al.  A Closed-Form Shave from Occam's Quantum Razor: Exact Results for Quantum Compression , 2015, ArXiv.

[74]  Herbert Jaeger,et al.  Observable Operator Models for Discrete Stochastic Time Series , 2000, Neural Computation.

[75]  Melville S. Green,et al.  Markoff Random Processes and the Statistical Mechanics of Time-Dependent Phenomena , 1952 .