Reconstruction of epsilon-Machines in Predictive Frameworks and Decisional States

This article introduces both a new algorithm for reconstructing epsilon-machines from data, as well as the decisional states. These are defined as the internal states of a system that lead to the same decision, based on a user-provided utility or pay-off function. The utility function encodes some a priori knowledge external to the system, it quantifies how bad it is to make mistakes. The intrinsic underlying structure of the system is modeled by an epsilon-machine and its causal states. The decisional states form a partition of the lower-level causal states that is defined according to the higher-level user's knowledge. In a complex systems perspective, the decisional states are thus the "emerging" patterns corresponding to the utility function. The transitions between these decisional states correspond to events that lead to a change of decision. The new REMAPF algorithm estimates both the epsilon-machine and the decisional states from data. Application examples are given for hidden model reconstruction, cellular automata filtering, and edge detection in images.

[1]  Andrew McCallum,et al.  Instance-Based Utile Distinctions for Reinforcement Learning with Hidden State , 1995, ICML.

[2]  C. Moore,et al.  Automatic filters for the detection of coherent structure in spatiotemporal systems. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Fabio Boschetti,et al.  Application of Computational Mechanics to the Analysis of Seismic Time-series via Numerical Optimisation , 2008, New Generation Computing.

[4]  Gilles Celeux,et al.  Selecting hidden Markov model state number with cross-validated likelihood , 2008, Comput. Stat..

[5]  Nihat Ay,et al.  On the Generative Nature of Prediction , 2009, Adv. Complex Syst..

[6]  Nicolas Brodu,et al.  A synthesis and a practical approach to complex systems , 2006, Complex..

[7]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[8]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[9]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[10]  Tarn Dunong.  Bandwidth selectors for multivariate kernel density estimation , 2005, Bulletin of the Australian Mathematical Society.

[11]  Suyash P. Awate,et al.  Adaptive, nonparametric markov models and information-theoretic methods for image restoration and segmentation , 2006 .

[12]  Anatole Lécuyer,et al.  Exploring two novel features for EEG-based brain-computer interfaces: Multifractal cumulants and predictive complexity , 2010, Neurocomputing.

[13]  Jaegwon Kim,et al.  Making Sense of Emergence , 1999 .

[14]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[15]  Don H. Johnson,et al.  Symmetrizing the Kullback-Leibler Distance , 2001 .

[16]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[17]  Wolfgang Löhr,et al.  Models of Discrete-Time Stochastic Processes and Associated Complexity Measures , 2009 .

[18]  Leslie Pack Kaelbling,et al.  Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[19]  Susanne Still,et al.  Information-theoretic approach to interactive learning , 2007, 0709.1948.

[20]  Gerik Scheuermann,et al.  Multifield visualization using local statistical complexity , 2007, IEEE Transactions on Visualization and Computer Graphics.

[21]  Erik Talvitie,et al.  Simple Local Models for Complex Dynamical Systems , 2008, NIPS.

[22]  Marco Wiering,et al.  Utile distinction hidden Markov models , 2004, ICML.

[23]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[24]  Cosma Rohilla Shalizi,et al.  Blind Construction of Optimal Nonlinear Recursive Predictors for Discrete Sequences , 2004, UAI.

[25]  R. Bellman,et al.  Dynamic Programming and Markov Processes , 1960 .

[26]  Robert Givan,et al.  Relational Reinforcement Learning: An Overview , 2004, ICML 2004.

[27]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[28]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[29]  S. Wolfram.  Statistical mechanics of cellular automata , 1983 .

[30]  Herbert Jaeger,et al.  Observable Operator Models for Discrete Stochastic Time Series , 2000, Neural Computation.

[31]  C. Shalizi,et al.  Causal architecture, complexity and self-organization in time series and cellular automata , 2001 .

[32]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[33]  Cosma Rohilla Shalizi,et al.  Methods and Techniques of Complex Systems Science: An Overview , 2003, nlin/0307015.

[34]  P. Bühlmann,et al.  Variable Length Markov Chains: Methodology, Computing, and Software , 2004 .

[35]  Douglas Lind,et al.  An Introduction to Symbolic Dynamics and Coding , 1995 .

[36]  Asok Ray,et al.  Symbolic dynamic analysis of complex systems for anomaly detection , 2004, Signal Process..