On the Complexity of Computing Maximum Entropy for Markovian Models

We investigate the complexity of computing entropy of various Markovian models including Markov Chains (MCs), Interval Markov Chains (IMCs) and Markov Decision Processes (MDPs). We consider both entropy and entropy rate for general MCs, and study two algorithmic questions, i.e., entropy approximation problem and entropy threshold problem. The former asks for an approximation of the entropy/entropy rate within a given precision, whereas the latter aims to decide whether they exceed a given threshold. We give polynomial-time algorithms for the approximation problem, and show the threshold problem is in P CH3 (hence in PSPACE) and in P assuming some number-theoretic conjectures. Furthermore, we study both questions for IMCs and MDPs where we aim to maximise the entropy/entropy rate among an infinite family of MCs associated with the given model. We give various conditional decidability results for the threshold problem, and show the approximation problem is solvable in polynomial-time via convex programming.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[2]  Mahesh Viswanathan,et al.  Model-Checking Markov Chains in the Presence of Uncertainties , 2006, TACAS.

[3]  Axel Legay,et al.  Maximizing entropy over Markov processes , 2014, J. Log. Algebraic Methods Program..

[4]  Nicolas Basset.  A Maximal Entropy Stochastic Process for a Timed Automaton, , 2013, ICALP.

[5]  Taolue Chen,et al.  On the complexity of model checking interval-valued discrete time Markov chains , 2013, Inf. Process. Lett..

[6]  Axel Legay,et al.  Quantifying information leakage of randomized protocols , 2015, Theor. Comput. Sci..

[7]  Angus Macintyre,et al.  On the decidability of the real exponential field , 1996 .

[8]  V. Girardin.  Entropy Maximization for Markov and Semi-Markov Processes , 2004 .

[9]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[10]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[11]  L. Lovász,et al.  Geometric Algorithms and Combinatorial Optimization , 1981 .

[12]  Alberto L. Sangiovanni-Vincentelli,et al.  Polynomial-Time Verification of PCTL Properties of MDPs with Convex Uncertainties , 2013, CAV.

[13]  James Worrell,et al.  LTL Model Checking of Interval Markov Chains , 2013, TACAS.

[14]  Pavol Cerný,et al.  The Complexity of Quantitative Information Flow Problems , 2011, 2011 IEEE 24th Computer Security Foundations Symposium.

[15]  Lev V. Utkin,et al.  Interval-Valued Finite Markov Chains , 2002, Reliab. Comput..

[16]  Rohit Chadha,et al.  The Complexity of Quantitative Information Flow in Recursive Programs , 2012, FSTTCS.

[17]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[18]  Michele Boreale.  Quantifying information leakage in process calculi , 2009, Inf. Comput..

[19]  Peter Bro Miltersen,et al.  2 The Task of a Numerical Analyst , 2022 .

[20]  Krishnendu Chatterjee,et al.  Model-Checking omega-Regular Properties of Interval Markov Chains , 2008, FoSSaCS.

[21]  Kousha Etessami,et al.  A Note on the Complexity of Comparing Succinctly Represented Integers, with an Application to Maximum Probability Parsing , 2013, TOCT.

[22]  Luca de Alfaro,et al.  Computing Minimum and Maximum Reachability Times in Probabilistic Systems , 1999, CONCUR.

[23]  N. Biggs.  GEOMETRIC ALGORITHMS AND COMBINATORIAL OPTIMIZATION: (Algorithms and Combinatorics 2) , 1990 .

[24]  A. Wilkie.  Model completeness results for expansions of the ordered field of real numbers by restricted Pfaffian functions and the exponential function , 1996 .

[25]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[26]  Hirotoshi Yasuoka,et al.  Quantitative Information Flow - Verification Hardness and Possibilities , 2010, 2010 23rd IEEE Computer Security Foundations Symposium.

[27]  W. Parry.  Intrinsic Markov chains , 1964 .

[28]  Axel Legay,et al.  QUAIL: A Quantitative Security Analyzer for Imperative Code , 2013, CAV.

[29]  Helmut Veith,et al.  Parameterized Verification of Asynchronous Shared-Memory Systems , 2013, Lecture Notes in Computer Science.