Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization

•A sequential sampling algorithm for the infinite-horizon approximate dynamic programming is proposed.•A new stopping criterion to effectively identify an optimally equivalent value function is given.•The extrapolation issue of approximate value function built by MARS is explored and discussed. The...

Full description

Saved in:
Bibliographic Details
Published in:Computers & operations research Vol. 124; p. 105032
Main Authors: Chen, Ying, Liu, Feng, Rosenberger, Jay M., Chen, Victoria C.P., Kulvanitchaiyanunt, Asama, Zhou, Yuan
Format: Journal Article
Language:English
Published: New York Elsevier Ltd 01.12.2020
Pergamon Press Inc
Subjects:
Algorithms
Approximate dynamic programming
Approximation
Criteria
Decision making
Design analysis
Dynamic programming
Extrapolation
Horizon
Iterative methods
Operations research
Optimization
Regression analysis
Sequential sampling
Spline functions
State space sampling
Stopping criterion
Extrapolation
Approximate dynamic programming
State space sampling
Stopping criterion
ISSN:0305-0548, 0305-0548
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•A sequential sampling algorithm for the infinite-horizon approximate dynamic programming is proposed.•A new stopping criterion to effectively identify an optimally equivalent value function is given.•The extrapolation issue of approximate value function built by MARS is explored and discussed. The approximate dynamic programming (ADP) method based on the design and analysis of computer experiments (DACE) approach has been demonstrated as an effective method to solve multistage decision-making problems in the literature. However, this method is still not efficient for infinite-horizon optimization considering the required large volume of sampling in the state space and high-quality value function identification. Therefore, we propose a sequential sampling algorithm and embed it into a DACE-based ADP method to obtain a high-quality value function approximation. Considering the limitations of the traditional stopping criterion (Bellman error bound), we further propose a 45-degree line stopping criterion to terminate value iteration early by identifying an optimally equivalent value function. A comparison of the computational results with those of other three existing policies indicates that the proposed sampling algorithm and stopping criterion can determine a high-quality ADP policy. Finally, we discuss the extrapolation issue of the value function approximated by multivariate adaptive regression splines, the results of which further demonstrate the quality of the ADP policy generated in this study.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0305-0548
0305-0548
DOI:10.1016/j.cor.2020.105032