Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation

Reinforcement learning (RL) has achieved tremendous success as a general framework for learning how to make decisions. However, this success relies on the interactive hand-tuning of a reward function by RL experts. On the other hand, inverse reinforcement learning (IRL) seeks to learn a reward funct...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2020 15th ACM/IEEE International Conference on Human-Robot Interaction (HRI) s. 659 - 668
Hlavní autori: Chen, Letian, Paleja, Rohan, Ghuy, Muyleng, Gombolay, Matthew
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: New York, NY, USA ACM 09.03.2020
Edícia:ACM Conferences
Predmet:
Computational modeling
Computing methodologies > Machine learning > Learning settings > Learning from demonstrations
Computing methodologies > Machine learning > Machine learning approaches > Neural networks
Heterogeneous Demonstration
Human-robot interaction
Inference algorithms
Inverse Reinforcement Learning
Neural networks
Preference Learning
Reinforcement learning
Task analysis
Theory of computation > Theory and algorithms for application domains > Machine learning theory > Reinforcement learning > Inverse reinforcement learning
Trajectory
inverse reinforcement learning
heterogeneous demonstration
preference learning
ISBN:1450367461, 9781450367462
ISSN:2167-2148
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Reinforcement learning (RL) has achieved tremendous success as a general framework for learning how to make decisions. However, this success relies on the interactive hand-tuning of a reward function by RL experts. On the other hand, inverse reinforcement learning (IRL) seeks to learn a reward function from readily-obtained human demonstrations. Yet, IRL suffers from two major limitations: 1) reward ambiguity - there are an infinite number of possible reward functions that could explain an expert's demonstration and 2) heterogeneity - human experts adopt varying strategies and preferences, which makes learning from multiple demonstrators difficult due to the common assumption that demonstrators seeks to maximize the same reward. In this work, we propose a method to jointly infer a task goal and humans' strategic preferences via network distillation. This approach enables us to distill a robust task reward (addressing reward ambiguity) and to model each strategy's objective (handling heterogeneity). We demonstrate our algorithm can better recover task reward and strategy rewards and imitate the strategies in two simulated tasks and a real-world table tennis task.
ISBN:1450367461
9781450367462
ISSN:2167-2148
DOI:10.1145/3319502.3374791