A Formalism for Stochastic Decision Processes with Asynchronous Events

We present the generalized semi-Markov decision process (GSMDP) as a natural model for stochastic decision processes with asynchronous events in hope to spur interest in asynchronous models, often overlooked in AI literature.

[1]  L. Cantaluppi,et al.  Optimality of Piecewise-Constant Policies in Semi-Markov Decision Chains , 1984 .

[2]  Miklós Telek,et al.  Acyclic discrete phase type distributions: properties and a parameter estimation algorithm , 2003, Perform. Evaluation.

[3]  Håkan L. S. Younes,et al.  Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions , 2004, AAAI.

[4]  P. Glynn.  A GSMP formalism for discrete event systems , 1989, Proc. IEEE.

[5]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[6]  Carlos Guestrin,et al.  Multiagent Planning with Factored MDPs , 2001, NIPS.

[7]  Marcel F. Neuts,et al.  Matrix-geometric solutions in stochastic models - an algorithmic approach , 1982 .

[8]  Gerald S. Shedler.  Regenerative Stochastic Simulation , 1992 .

[9]  Paul Adrien Maurice Dirac,et al.  The physical interpretation of the quantum dynamics , 1927 .

[10]  Rajeev Alur,et al.  Model-Checking in Dense Real-time , 1993, Inf. Comput..

[11]  S. S. Chitgopekar.  Continuous Time Markovian Sequential Control Processes , 1969 .

[12]  Sridhar Mahadevan,et al.  Decision-Theoretic Planning with Concurrent Temporally Extended Actions , 2001, UAI.

[13]  Lawrence D. Stone,et al.  Necessary and Sufficient Conditions for Optimal Control of Semi-Markov Jump Processes , 1973 .

[14]  Geoffrey J. Gordon.  Stable Function Approximation in Dynamic Programming , 1995, ICML.

[15]  Mausam,et al.  Solving Concurrent Markov Decision Processes , 2004, AAAI.