Direct heuristic dynamic programming based on an improved PID neural network

In this paper, an improved PID-neural network （IPIDNN） structure is proposed and applied to the critic and action networks of direct heuristic dynamic programming （DHDP）. As one of online learning algorithm of approximate dynamic programming （ADP）, DHDP has demonstrated its applicability to large st...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of control theory and applications Jg. 10; H. 4; S. 497 - 503
Hauptverfasser: Sun, Jian, Liu, Feng, Si, Jennie, Mei, Shengwei
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Heidelberg South China University of Technology and Academy of Mathematics and Systems Science, CAS 01.11.2012
Department of Electrical Engineering, Tsinghua University, Beijing 100084, China%Department of Electrical Engineering, Arizona State University, Tempe AZ 85287-5706, USA
Schlagworte:
Algorithms
Brief Paper
Complexity
Computational Intelligence
Control
Control and Systems Theory
Control systems
Dynamic programming
Engineering
Heuristic
Learning
Mathematical analysis
Mechatronics
Networks
Neural networks
Optimization
PID神经网络
Robotics
Systems Theory
启发式动态规划
在线学习算法
基础
最优性方程
状态反馈
系统验证
输出反馈控制
Approximate dynamic programming (ADP)
Direct heuristic dynamic programming (DHDP)
Improved PID neural network (IPIDNN)
ISSN:1672-6340, 1993-0623
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, an improved PID-neural network （IPIDNN） structure is proposed and applied to the critic and action networks of direct heuristic dynamic programming （DHDP）. As one of online learning algorithm of approximate dynamic programming （ADP）, DHDP has demonstrated its applicability to large state and control problems. Theoretically, the DHDP algorithm requires access to full state feedback in order to obtain solutions to the Bellman optimality equation. Unfortunately, it is not always possible to access all the states in a real system. This paper proposes a solution by suggesting an IPIDNN configuration to construct the critic and action networks to achieve an output feedback control. Since this structure can estimate the integrals and derivatives of measurable outputs, more system states are utilized and thus better control performance are expected. Compared with traditional PIDNN, this configuration is flexible and easy to expand. Based on this structure, a gradient decent algorithm for this IPIDNN-based DHDP is presented. Convergence issues are addressed within a single learning time step and for the entire learning process. Some important insights are provided to guide the implementation of the algorithm. The proposed learning controller has been applied to a cart-pole system to validate the effectiveness of the structure and the algorithm.
Bibliographie:Approximate dynamic programming （ADP）; Direct heuristic dynamic programming （DHDP）; ImprovedPID neural network （IPIDNN）
In this paper, an improved PID-neural network （IPIDNN） structure is proposed and applied to the critic and action networks of direct heuristic dynamic programming （DHDP）. As one of online learning algorithm of approximate dynamic programming （ADP）, DHDP has demonstrated its applicability to large state and control problems. Theoretically, the DHDP algorithm requires access to full state feedback in order to obtain solutions to the Bellman optimality equation. Unfortunately, it is not always possible to access all the states in a real system. This paper proposes a solution by suggesting an IPIDNN configuration to construct the critic and action networks to achieve an output feedback control. Since this structure can estimate the integrals and derivatives of measurable outputs, more system states are utilized and thus better control performance are expected. Compared with traditional PIDNN, this configuration is flexible and easy to expand. Based on this structure, a gradient decent algorithm for this IPIDNN-based DHDP is presented. Convergence issues are addressed within a single learning time step and for the entire learning process. Some important insights are provided to guide the implementation of the algorithm. The proposed learning controller has been applied to a cart-pole system to validate the effectiveness of the structure and the algorithm.
44-1600/TP
ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ISSN:1672-6340
1993-0623
DOI:10.1007/s11768-012-0112-0