Distributed Reinforcement Learning using Bi-directional Decision Making for Controlling Multi-Stage Flow Systems

Autonomous control systems have been requested recently for large-scale real systems. Distributed reinforcement learning is attracting attention specifically in control of physical flow systems such as lifeline systems. In this paper, we will introduce a model of Multi-Stage Flow System (MSFS) as a new problem class. MSFS is a framework which can describe various physical flow systems. Furthermore, it is effective in handling balance between a purpose of system and constraints, constraints under uncertainty and so on that are difficult to solve in conventional methods because of its features. We propose a new bi-directional decision making algorithm with feasible action sets based on a least commitment strategy. We apply our method to controlling of real sewerage systems. The simulation results show that only our method satisfies permissible levels and attains the performance within an acceptance level.