A new approximate dynamic programming algorithm based on an actor-critic framework for optimal control of alkali-surfactant-polymer flooding

An approximate dynamic programming algorithm based on an actor-critic framework is proposed in this article. This algorithm takes the actor-critic framework as the basic framework, in which the actor and the critic are used to approximate the optimal value function and the control strategy, respecti...

Full description

Saved in:
Bibliographic Details
Published in:Engineering optimization Vol. 51; no. 12; pp. 2147 - 2168
Main Authors: Li, Shurong, Han, Lu, Ge, Yulei, Shi, Yuhuan
Format: Journal Article
Language:English
Published: Abingdon Taylor & Francis 02.12.2019
Taylor & Francis Ltd
Subjects:
action weighting
actor-critic framework
Algorithms
Approximate dynamic programming
ASP flooding
Basis functions
Dynamic programming
eligibility trace
Optimal control
Parameters
Polymer flooding
Weighting methods
ISSN:0305-215X, 1029-0273
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An approximate dynamic programming algorithm based on an actor-critic framework is proposed in this article. This algorithm takes the actor-critic framework as the basic framework, in which the actor and the critic are used to approximate the optimal value function and the control strategy, respectively. At first, the linear basis function approximator is used to approximate the value function. Then the method of basis function construction based on system characteristics is introduced. Furthermore, since the injection concentration of ASP flooding has a fixed interval, the action weighting method is adopted to restrict and approximate the optimal control action. The value function parameter and the two strategy parameters are updated by the gradient descent method. Meanwhile the eligibility trace is introduced to accelerate convergence. Finally, ASP flooding with four injection wells and nine production wells is used to test the effect of the proposed method.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0305-215X
1029-0273
DOI:10.1080/0305215X.2019.1570180