Adaptive Dynamic Programming for Optimal Control of Unknown LTI System via Interval Excitation

In this article, we investigate the optimal control problem for an unknown linear time-invariant system. To solve this problem, a novel composite policy iteration algorithm based on adaptive dynamic programming is developed to adaptively learn the optimal control policy from system data. The existin...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control Vol. 70; no. 7; pp. 4896 - 4903
Main Authors: Ma, Yong-Sheng, Sun, Jian, Xu, Yong, Cui, Shi-Sheng, Wu, Zheng-Guang
Format: Journal Article
Language:English
Published: New York IEEE 01.07.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Adaptation models
Adaptive algorithms
Adaptive dynamic programming (ADP)
Adaptive systems
Approximation algorithms
Control systems
Convergence
Data storage
Dynamic programming
Excitation
Heuristic algorithms
Iterative algorithms
Linear systems
Memory
Optimal control
policy iteration (PI)
Time invariant systems
Training
ISSN:0018-9286, 1558-2523
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this article, we investigate the optimal control problem for an unknown linear time-invariant system. To solve this problem, a novel composite policy iteration algorithm based on adaptive dynamic programming is developed to adaptively learn the optimal control policy from system data. The existing methods require the initial stabilizing control policy, the persistence of excitation (PE) condition and the data storage to ensure the algorithm convergence. Fundamentally different from them, these restrictions can be relaxed in the proposed method. Specifically, an adaptive parameter is elaborately designed to remove the requirement of the initial stabilizing control policy. Besides, an online data calculation scheme is proposed, which cannot only replace the stored historical data by online data, but also can relax the PE condition to the interval excitation condition. The simulation results demonstrate the efficacy of the proposed algorithm, and its superiority is also demonstrated by comparing it with existing algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2025.3542328