Reward criteria impact on the performance of reinforcement learning agent for autonomous navigation

In reinforcement learning, an agent takes action at every time step (follows a policy) in an environment to maximize the expected cumulative reward. Therefore, the shaping of a reward function plays a crucial role in an agent’s learning. Designing an optimal reward function is not a trivial task. In...

Full description

Saved in:
Bibliographic Details
Published in:Applied soft computing Vol. 126; p. 109241
Main Authors: Dayal, Aveen, Cenkeramaddi, Linga Reddy, Jha, Ajit
Format: Journal Article
Language:English
Published: Elsevier B.V 01.09.2022
Subjects:
Autonomous navigation
Deep reinforcement learning
Machine learning and artificial intelligence
Reward criteria
Deep reinforcement learning
Autonomous navigation
Reward criteria
Machine learning and artificial intelligence
ISSN:1568-4946, 1872-9681
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In reinforcement learning, an agent takes action at every time step (follows a policy) in an environment to maximize the expected cumulative reward. Therefore, the shaping of a reward function plays a crucial role in an agent’s learning. Designing an optimal reward function is not a trivial task. In this article, we propose a reward criterion using which we develop different reward functions. The reward criterion chosen is based on the percentage of positive and negative rewards received by an agent. This reward criteria further gives rise to three different classes, ‘Balanced Class,’ ‘Skewed Positive Class,’ and ‘Skewed Negative Class.’ We train a Deep Q-Network agent on a point-goal based navigation task using the different reward classes. We also compare the performance of the proposed classes with a benchmark class. Based on the experiments, the skewed negative class outperforms the benchmark class by achieving very less variance. On the other hand, the benchmark class converges relatively faster than the skewed negative class. •A reward criterion to assess the performance of an RL agent.•Various reward functions to train an RL agent.•The proportion of positive and negative rewards in a reward shaping function.•The reward criterion: ‘Balanced Class’, ‘Skewed Positive Class’ and ‘Skewed Negative Class’.•The performance of an RL agent in the case of autonomous navigation.
ISSN:1568-4946
1872-9681
DOI:10.1016/j.asoc.2022.109241