A new approximate dynamic programming algorithm based on an actor–critic framework for optimal control of alkali–surfactant–polymer flooding

ABSTRACT An approximate dynamic programming algorithm based on an actor–critic framework is proposed in this article. This algorithm takes the actor–critic framework as the basic framework, in which the actor and the critic are used to approximate the optimal value function and the control strategy, respectively. At first, the linear basis function approximator is used to approximate the value function. Then the method of basis function construction based on system characteristics is introduced. Furthermore, since the injection concentration of ASP flooding has a fixed interval, the action weighting method is adopted to restrict and approximate the optimal control action. The value function parameter and the two strategy parameters are updated by the gradient descent method. Meanwhile the eligibility trace is introduced to accelerate convergence. Finally, ASP flooding with four injection wells and nine production wells is used to test the effect of the proposed method.