Planning large systems with MDPs: case study of inland waterways supervision

Inland waterway management is likely to go through heavy changes due to an expected traffic increase in a context of climate change. Those changes will require an adaptive and resilient management of the water resource. The aim is to have an optimal plan for the distribution of the water resource on the whole inland waterway network, while taking into account the uncertainties arising from the operations of such a network. A representative model using Markov decision processes is proposed to model the dynamic and the uncertainties of the waterways. The proposed model is able to coordinate multiple entities over multiple time steps in order to prevent an overflow of a test network. However, this model suffers from a lack of scalability and is unable to represent real case applications. Advantages and limitations of several approaches of the literature to circumvent this limitation are discussed according to our case study.

[1]  Shobha Venkataraman,et al.  Context-specific multiagent coordination and planning with factored MDPs , 2002, AAAI/IAAI.

[2]  P. Hawkes,et al.  Climate change and navigation : waterborne transport, ports and waterways: a review of climate change drivers, impacts responses and mitigation , 2010 .

[3]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[4]  Craig Boutilier,et al.  Exploiting Structure in Policy Construction , 1995, IJCAI.

[5]  B. Bates,et al.  Climate change and water. , 2008 .

[6]  B. Arkell,et al.  Impact of climate change on London's transport network , 2006 .

[7]  Arnaud Doniec,et al.  Dynamic optimization approaches for resource allocation planning in inland navigation networks , 2016, 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA).

[8]  Mirjana Golušin,et al.  Policy and promotion of sustainable inland waterway transport in Europe ― Danube River , 2011 .

[9]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[10]  Jesse Hoey,et al.  SPUDD: Stochastic Planning using Decision Diagrams , 1999, UAI.

[11]  Thomas Dean,et al.  Decomposition Techniques for Planning in Stochastic Domains , 1995, IJCAI.

[12]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[13]  R. Bellman.  A Markovian Decision Process , 1957 .

[14]  C. Brand,et al.  The UK transport carbon model: An integrated life cycle approach to explore low carbon futures , 2012 .

[15]  François Charpillet,et al.  A heuristic approach for solving decentralized-POMDP: assessment on the pursuit problem , 2002, SAC '02.

[16]  Arnaud Doniec,et al.  Efficient management of inland navigation reaches equipped with lift pumps in a climate change context , 2016 .

[17]  R. Dekker,et al.  The impact of greening on supply chain design and cost: a case for a developing region , 2012 .

[18]  Arnaud Doniec,et al.  Study of drought impact on inland navigation systems based on a flow network model , 2015, 2015 XXV International Conference on Information, Communication and Automation Technologies (ICAT).

[19]  Ronald Parr,et al.  Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems , 1998, UAI.

[20]  Lounis Adouane,et al.  Punctual versus continuous auction coordination for multi-robot and multi-task topological navigation , 2016, Auton. Robots.

[21]  Bart Jourquin,et al.  Estimating the impacts of water depth and new infrastructures on transport by inland navigation: a multimodal approach for the Rhine corridor , 2012 .

[22]  Carlos Guestrin,et al.  Multiagent Planning with Factored MDPs , 2001, NIPS.

[23]  Bart Jourquin,et al.  Climate change impacts on transport on the Rhine and Danube: A multimodal approach , 2014 .

[24]  N. Wanders,et al.  Human and climate impacts on the 21st century hydrological drought , 2015 .

[25]  Régis Sabbadin,et al.  Graph partitioning techniques for Markov Decision Processes decomposition , 2002, ECAI.

[26]  Makoto Yokoo,et al.  Networked Distributed POMDPs: A Synergy of Distributed Constraint Optimization and POMDPs , 2005, IJCAI.