Multi-weighted Markov Decision Processes with Reachability Objectives

In this paper, we are interested in the synthesis of schedulers in double-weighted Markov decision processes, which satisfy both a percentile constraint over a weighted reachability condition, and a quantitative constraint on the expected value of a random variable defined using a weighted reachability condition. This problem is inspired by the modelization of an electric-vehicle charging problem. We study the cartography of the problem, when one parameter varies, and show how a partial cartography can be obtained via two sequences of opimization problems. We discuss completeness and feasability of the method.

[1]  Daniel R. Jiang,et al.  Practicality of Nested Risk Measures for Dynamic Electric Vehicle Charging , 2016, 1605.02848.

[2]  Patricia Bouyer,et al.  Stratégies d'ordonnancement de consommation d'énergie en présence d'information imparfaite de prévision , 2017 .

[3]  Mickael Randour,et al.  Variations on the Stochastic Shortest Path Problem , 2014, VMCAI.

[4]  Christoph Haase,et al.  The Odds of Staying on Budget , 2014, ICALP.

[5]  Jan J. M. M. Rutten,et al.  Stream Differential Equations: Specification Formats and Solution Methods , 2014, Log. Methods Comput. Sci..

[6]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[7]  Jan Kretínský,et al.  Conditional Value-at-Risk for Reachability and Mean Payoff in Markov Decision Processes , 2018, LICS.

[8]  Mickael Randour,et al.  Threshold Constraints with Guarantees for Parity Objectives in Markov Decision Processes , 2017, ICALP.

[9]  Christel Baier,et al.  Trade-off analysis meets probabilistic model checking , 2014, CSL-LICS.

[10]  Samson Lasaulce,et al.  Reducing the Impact of EV Charging Operations on the Distribution Network , 2016, IEEE Transactions on Smart Grid.

[11]  Krishnendu Chatterjee,et al.  Markov Decision Processes with Multiple Long-Run Average Objectives , 2007, FSTTCS.

[12]  Krishnendu Chatterjee,et al.  Unifying Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2015, 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science.

[13]  Stefano Berardi,et al.  Knowledge Spaces and the Completeness of Learning Strategies , 2014, Log. Methods Comput. Sci..

[14]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2014, CAV.

[15]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[16]  Christel Baier,et al.  Stochastic Shortest Paths and Weight-Bounded Properties in Markov Decision Processes , 2018, LICS.

[17]  Lorenzo Clemente,et al.  Multidimensional beyond Worst-Case and Almost-Sure Problems for Mean-Payoff Objectives , 2015, 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science.

[18]  Christel Baier,et al.  Maximizing the Conditional Expected Reward for Reaching the Goal , 2017, TACAS.

[19]  Jonathan Donadee,et al.  Stochastic Optimization of Grid to Vehicle Frequency Regulation Capacity Bids , 2014, IEEE Transactions on Smart Grid.