MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems

In this paper, the first probably approximately correct (PAC) algorithm for continuous deterministic systems without relying on any system dynamics is proposed. It combines the state aggregation technique and the efficient exploration principle, and makes high utilization of online observed samples....

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transaction on neural networks and learning systems Ročník 26; číslo 2; s. 346 - 356
Hlavní autoři: Zhao, Dongbin, Zhu, Yuanheng
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States IEEE 01.02.2015
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithm design and analysis
Algorithms
Approximation algorithms
Computer simulation
Dynamical systems
Dynamics
Efficient exploration
Exploration
Heuristic algorithms
Learning
Learning systems
Mathematical analysis
Neural networks
Partitioning algorithms
Policies
Polynomials
probably approximately correct (PAC)
reinforcement learning (RL)
state aggregation
Upper bound
Efficient exploration
probably approximately correct (PAC)
state aggregation
reinforcement learning (RL)
ISSN:2162-237X, 2162-2388, 2162-2388
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, the first probably approximately correct (PAC) algorithm for continuous deterministic systems without relying on any system dynamics is proposed. It combines the state aggregation technique and the efficient exploration principle, and makes high utilization of online observed samples. We use a grid to partition the continuous state space into different cells to save samples. A near-upper Q operator is defined to produce a near-upper Q function using samples in each cell. The corresponding greedy policy effectively balances between exploration and exploitation. With the rigorous analysis, we prove that there is a polynomial time bound of executing nonoptimal actions in our algorithm. After finite steps, the final policy reaches near optimal in the framework of PAC. The implementation requires no knowledge of systems and has less computation complexity. Simulation studies confirm that it is a better performance than other similar PAC algorithms.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2014.2371046