Robustness Assessment of Asynchronous Advantage Actor-Critic Based on Dynamic Skewness and Sparseness Computation: A Parallel Computing View

Reinforcement learning as autonomous learning is greatly driving artificial intelligence (AI) development to practical applications. Having demonstrated the potential to significantly improve synchronously parallel learning, the parallel computing based asynchronous advantage actor-critic (A3C) open...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of computer science and technology Jg. 36; H. 5; S. 1002 - 1021
Hauptverfasser: Chen, Tong, Liu, Ji-Qiang, Li, He, Wang, Shuo-Ru, Niu, Wen-Jia, Tong, En-Dong, Chang, Liang, Chen, Qi Alfred, Li, Gang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Singapore Springer Singapore 01.10.2021
Springer
Springer Nature B.V
Beijing Key Laboratory of Security and Privacy in Intelligent Transportation,Beijing Jiaotong University Beijing 100044,China%Guangxi Key Laboratory of Trusted Software,Guilin University of Electronic Technology,Guilin 541004,China%Donald Bren School of Information and Computer Sciences,University of California,Irvine 92697,U.S.A.%Centre for Cyber Security Research and Innovation,Deakin University,Geelong,VIC 3216,Australia
Schlagworte:
Algorithms
Artificial Intelligence
Computer Science
Data Structures and Information Theory
Information Systems Applications (incl.Internet)
Machine learning
Regular Paper
Robustness (mathematics)
Skewness
Software Engineering
Theory of Computation
sparseness
skewness
robustness assessment
asynchronous advantage actor-critic
reinforcement learning
ISSN:1000-9000, 1860-4749
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Reinforcement learning as autonomous learning is greatly driving artificial intelligence (AI) development to practical applications. Having demonstrated the potential to significantly improve synchronously parallel learning, the parallel computing based asynchronous advantage actor-critic (A3C) opens a new door for reinforcement learning. Unfortunately, the acceleration's inuence on A3C robustness has been largely overlooked. In this paper, we perform the first robustness assessment of A3C based on parallel computing. By perceiving the policy’s action, we construct a global matrix of action probability deviation and define two novel measures of skewness and sparseness to form an integral robustness measure. Based on such static assessment, we then develop a dynamic robustness assessing algorithm through situational whole-space state sampling of changing episodes. Extensive experiments with different combinations of agent number and learning rate are implemented on an A3C-based pathfinding application, demonstrating that our proposed robustness assessment can effectively measure the robustness of A3C, which can achieve an accuracy of 83.3%.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1000-9000
1860-4749
DOI:10.1007/s11390-021-1217-z