A Note on Quantum Markov Models

The study of Markov models is central to control theory and machine learning. A quantum analogue of partially observable Markov decision process was studied in (Barry, Barry, and Aaronson, Phys. Rev. A, 90, 2014). It was proved that goal-state reachability is undecidable in the quantum setting, whereas it is decidable classically. In contrast to this classical-to-quantum transition from decidable to undecidable, we observe that the problem of approximating the optimal policy which maximizes the average discounted reward over an infinite horizon remains decidable in the quantum setting. Given that most relevant problems related to Markov decision process are undecidable classically (which immediately implies undecidability in the quantum case), this provides one of the few examples where the quantum problem is tractable.

[1]  J Eisert,et al.  Quantum measurement occurrence is undecidable. , 2011, Physical review letters.

[2]  John Watrous,et al.  The Theory of Quantum Information , 2018 .

[3]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[4]  K. Wiesner,et al.  Hidden Quantum Markov Models and non-adaptive read-out of many-body states , 2010, 1002.2337.

[5]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[6]  M. Szegedy,et al.  Quantum Walk Based Search Algorithms , 2008, TAMC.

[7]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs , 1978, Oper. Res..

[8]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[9]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[10]  Jennifer L. Barry,et al.  Quantum partially observable Markov decision processes , 2014 .

[11]  James P. Crutchfield,et al.  Computation in Finitary Stochastic and Quantum Processes , 2006 .

[12]  I. Chuang,et al.  Quantum Computation and Quantum Information: Bibliography , 2010 .

[13]  F. Verstraete,et al.  The χ2-divergence and mixing times of quantum Markov processes , 2010, 1005.2358.

[14]  Vincent D. Blondel,et al.  Decidable and Undecidable Problems about Quantum Automata , 2005, SIAM J. Comput..

[15]  Anne Condon,et al.  On the undecidability of probabilistic planning and related stochastic optimization problems , 2003, Artif. Intell..

[16]  D. Blackwell.  Discounted Dynamic Programming , 1965 .

[17]  David Pérez-García,et al.  Are problems in Quantum Information Theory (un)decidable? , 2011, ArXiv.

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  Mark M. Wilde,et al.  Quantum Information Theory , 2013 .