An approximate dynamic programming method for the optimal control of Alkai-Surfactant-Polymer flooding

Abstract Since the complexity, coupling, distributed parameter, etc. of alkali-surfactant-polymer (ASP) flooding, common optimization methods cannot acquire the optimal solutions well. This paper brings an optimal control method for ASP flooding based on approximate dynamic programming (ADP). At first, take the net present value (NPV) as the performance index. Then the Actor-Critic algorithm based on gradient descent method is adopted to get the optimal injection strategy, in which Actor and Critic are used to approximate the control and value function, respectively. To improve the approximation performance, the linear approximation basis function based on system characteristic is constructed. Furthermore, to train and predict the control and value function in next step, a temporal difference (TD) learning algorithm is introduced to update the weight coefficients. Then, the control in ADP is generated according to the Gauss function and its weight is updated according to the sigmoid function of TD error, so that the optimal control can be searched. At last, the enhanced oil recovery problem of ASP flooding with four injection wells and nine production wells is solved by the proposed method to test the effect of proposed method.

[1]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[2]  Yang Lei,et al.  Optimization of ASP flooding based on dynamic scale IDP with mixed-integer , 2017 .

[3]  Zohreh Fathi,et al.  Optimal injection policies for enhanced oil recovery: Part 1-Theory and computational strategies , 1984 .

[4]  Phanish Suryanarayana,et al.  SPARC: Accurate and efficient finite-difference formulation and parallel implementation of Density Functional Theory: Isolated clusters , 2016, Comput. Phys. Commun..

[5]  Yang Lei,et al.  Spatial-Temporal ARX Modeling and Optimization for Polymer Flooding , 2014 .

[6]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[7]  Avi Seifert,et al.  Proper orthogonal decomposition of flow-field in non-stationary geometry , 2015, J. Comput. Phys..

[8]  Yang Lei,et al.  Optimal control of polymer flooding for enhanced oil recovery , 2013, Int. J. Model. Identif. Control..

[9]  Yang Lei,et al.  Optimal control for an alkali/surfactant/polymer flooding system , 2016, 2016 35th Chinese Control Conference (CCC).

[10]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[11]  Maurice Queyranne,et al.  Production and Inventory Model Using Net Present Value , 2002, Oper. Res..

[12]  Eduardo Gildin,et al.  Tensor based geology preserving reservoir parameterization with Higher Order Singular Value Decomposition (HOSVD) , 2016, Comput. Geosci..

[13]  Qiang Zhang,et al.  Optimal control of polymer flooding based on mixed-integer iterative dynamic programming , 2011, Int. J. Control.

[14]  Frank L. Lewis,et al.  A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems , 2013, Autom..

[15]  Shurong Li,et al.  A Novel Empirical Equation for Relative Permeability in Low Permeability Reservoirs , 2014 .

[16]  Bart De Schutter,et al.  Reinforcement Learning and Dynamic Programming Using Function Approximators , 2010 .

[17]  Nader Fathianpour,et al.  A hybrid simultaneous perturbation artificial bee colony and back-propagation algorithm for training a local linear radial basis neural network on ore grade estimation , 2017, Neurocomputing.

[18]  Chen An,et al.  Improved lumped models for transient combined convective and radiative cooling of multi-layer composite slabs , 2011 .

[19]  Renato Machado,et al.  Improved solution for node location multilateration algorithms in wireless sensor networks , 2016 .

[20]  Amir H. Mohammadi,et al.  Efficient screening of enhanced oil recovery methods and predictive economic analysis , 2014, Neural Computing and Applications.

[21]  Boudewijn P. F. Lelieveldt,et al.  Fast Automatic Step Size Estimation for Gradient Descent Optimization of Image Registration , 2016, IEEE Transactions on Medical Imaging.

[22]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Trans. Neural Networks Learn. Syst..

[23]  John N. Tsitsiklis,et al.  Neuro-dynamic programming: an overview , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.