Minimum memory for generating rare events.

We classify the rare events of structured, memoryful stochastic processes and use this to analyze sequential and parallel generators for these events. Given a stochastic process, we introduce a method to construct a process whose typical realizations are a given process' rare events. This leads to an expression for the minimum memory required to generate rare events. We then show that the recently discovered classical-quantum ambiguity of simplicity also occurs when comparing the structure of process fluctuations.

[1]  James P. Crutchfield,et al.  Occam’s Quantum Strop: Synchronizing and Compressing Classical Cryptic Processes via a Quantum Channel , 2015, Scientific Reports.

[2]  James P. Crutchfield,et al.  Signatures of Infinity: Nonergodicity and Resource Scaling in Prediction, Complexity, and Learning , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Lenard,et al.  Statistical Mechanics and Mathematical Problems , 1973 .

[4]  B. McMillan.  The Basic Theorems of Information Theory , 1953 .

[5]  A. Si,et al.  Entropy,Large Deviations,and Statistical Mechanics , 2011 .

[6]  Richard W Clarke,et al.  Application of computational mechanics to the analysis of natural data: an example in geomagnetism. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  Neri Merhav,et al.  On the estimation of the order of a Markov chain and universal data compression , 1989, IEEE Trans. Inf. Theory.

[8]  J. Crutchfield.  Between order and chaos , 2011, Nature Physics.

[9]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[10]  E. Lorenz,et al.  Short term fluctuations of wind and solar power systems , 2016, 1606.03426.

[11]  James P Crutchfield,et al.  Time's barbed arrow: irreversibility, crypticity, and stored information. , 2009, Physical review letters.

[12]  Daniel Ray Upper,et al.  Theory and algorithms for hidden Markov models and generalized hidden Markov models , 1998 .

[13]  Alexander B. Neiman,et al.  Characterizing the dynamics of stochastic bistable systems by measures of complexity , 1997 .

[14]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[15]  James P. Crutchfield,et al.  The Hidden Fragility of Complex Systems— Consequences of Change, Changing Consequences , 2020, 2003.11153.

[16]  Y. Tsuji,et al.  Similarity scaling of pressure fluctuation in turbulence. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  W. Marsden.  I and J , 2012 .

[18]  G. Parmigiani.  Large Deviation Techniques in Decision, Simulation and Estimation , 1992 .

[19]  Muhammad Sahimi,et al.  Turbulencelike behavior of seismic time series. , 2009, Physical review letters.

[20]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[21]  James P. Crutchfield,et al.  Computational Mechanics: Pattern and Prediction, Structure and Simplicity , 1999, ArXiv.

[22]  Nicholas F. Travers.  Bounds on Convergence of Entropy Rate Approximations in Hidden Markov Processes , 2013 .

[23]  Wolfgang Löhr.  Predictive models and generative complexity , 2012, J. Syst. Sci. Complex..

[24]  R. Mazo.  On the theory of brownian motion , 1973 .

[25]  T. Cover,et al.  A sandwich proof of the Shannon-McMillan-Breiman theorem , 1988 .

[26]  Muhammad Sahimi,et al.  Markov analysis and Kramers-Moyal expansion of nonstationary stochastic processes with application to the fluctuations in the oil price. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[28]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[29]  Klaus Lehnertz,et al.  Stochastic qualifiers of epileptic brain dynamics. , 2007, Physical review letters.

[30]  Ji-Ping Huang,et al.  Experimental econophysics: Complexity, self-organization, and emergent properties , 2015 .

[31]  Cina Aghamohammadi,et al.  Permutation approach, high frequency trading and variety of micro patterns in financial time series , 2014 .

[32]  DANNY CALEGARI,et al.  Thermodynamic Formalism , 2021, Lecture Notes in Mathematics.

[33]  James P. Crutchfield,et al.  Not All Fluctuations are Created Equal: Spontaneous Variations in Thermodynamic Function , 2016, ArXiv.

[34]  R. Durrett.  Probability: Theory and Examples , 1993 .

[35]  Raymond W. Yeung,et al.  Information Theory and Network Coding , 2008 .

[36]  S. Varadhan,et al.  Large deviations , 2019, Graduate Studies in Mathematics.

[37]  Peter Gmeiner,et al.  Equality conditions for internal entropies of certain classical and quantum models , 2011, 1108.5303.

[38]  Karoline Wiesner,et al.  A New Method for Inferring Hidden Markov Models from Noisy Time Sequences , 2012, PloS one.

[39]  R. Glen,et al.  Identifying and correcting non-Markov states in peptide conformational dynamics. , 2010, The Journal of chemical physics.

[40]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[41]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[42]  P. Mazur.  On the theory of brownian motion , 1959 .

[43]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[44]  Tamiki Komatsuzaki,et al.  Aggregated markov model using time series of single molecule dwell times with minimum excessive information. , 2013, Physical review letters.

[45]  Solomon Kullback,et al.  Information Theory and Statistics , 1970, The Mathematical Gazette.

[46]  J. Peinke,et al.  Stochastic analysis of different rough surfaces , 2004 .

[47]  J. Crutchfield.  The calculi of emergence: computation, dynamics and induction , 1994 .

[48]  J. Rogers.  Chaos , 1876 .

[49]  Nihat Ay,et al.  On the Generative Nature of Prediction , 2009, Adv. Complex Syst..

[50]  Ludwig Boltzmann,et al.  Lectures on Gas Theory , 1964 .

[51]  Brian H. Marcus,et al.  Analyticity of Entropy Rate of Hidden Markov Chains , 2005, IEEE Transactions on Information Theory.

[52]  J. Crutchfield,et al.  Fluctuation Spectroscopy , 1993 .

[53]  Ryan Tan,et al.  Towards quantifying complexity with quantum mechanics , 2014, 1404.6255.

[54]  Karoline Wiesner,et al.  Quantum mechanics can reduce the complexity of classical models , 2011, Nature Communications.

[55]  Ericka Stricklin-Parker,et al.  Ann , 2005 .

[56]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[57]  D. Vvedensky,et al.  Langevin equations for fluctuating surfaces. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[58]  Philip Sura,et al.  Stochastic Analysis of Southern and Pacific Ocean Sea Surface Winds , 2003 .

[59]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[60]  J. Peinke,et al.  Stochastic nature of series of waiting times. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[61]  M. R. Rahimi Tabar,et al.  Scale dependence of the directional relationships between coupled time series , 2013 .

[62]  T. Nagatani,et al.  Traffic jams induced by fluctuation of a leading car. , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[63]  L. Breiman.  The Individual Ergodic Theorem of Information Theory , 1957 .

[64]  James P. Crutchfield,et al.  A Closed-Form Shave from Occam's Quantum Razor: Exact Results for Quantum Compression , 2015, ArXiv.

[65]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[66]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[67]  L. Christophorou.  Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[68]  Christopher W. Fairall,et al.  Complexity in the atmosphere , 2000, IEEE Trans. Geosci. Remote. Sens..

[69]  Tom Kuusela,et al.  Stochastic heart-rate model can reveal pathologic cardiac dynamics. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[70]  H. Touchette.  The large deviation approach to statistical mechanics , 2008, 0804.0327.

[71]  Yoshitsugu Oono,et al.  Large Deviation and Statistical Physics , 1989 .

[72]  V. Climenhaga.  Markov chains and mixing times , 2013 .

[73]  Long-range correlation in cosmic microwave background radiation. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[74]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[75]  Mervyn P. Freeman,et al.  The application of computational mechanics to the analysis of geomagnetic data , 2001 .

[76]  Evgueni A. Haroutunian,et al.  Information Theory and Statistics , 2011, International Encyclopedia of Statistical Science.

[77]  R. Friedrich,et al.  Reconstruction of dynamical equations for traffic flow , 2002 .

[78]  Nihat Ay,et al.  Non-sufficient Memories That Are Sufficient for Prediction , 2009, Complex.

[79]  Dla Polski,et al.  EURO , 2004 .

[80]  Reynaldo D. Pinto,et al.  Inferring statistical complexity in the dripping faucet experiment , 1998 .

[81]  James P. Crutchfield,et al.  Prediction, Retrodiction, and the Amount of Information Stored in the Present , 2009, ArXiv.

[82]  James P. Crutchfield,et al.  Extreme Quantum Advantage when Simulating Strongly Coupled Classical Systems , 2016, ArXiv.

[83]  Patrick Milan,et al.  Kolmogorov spectrum of renewable wind and solar power fluctuations , 2014 .

[84]  James P. Crutchfield,et al.  The Ambiguity of Simplicity , 2016, ArXiv.