Learned graphical models for probabilistic planning provide a new class of movement primitives

BIOLOGICAL MOVEMENT GENERATION COMBINES THREE INTERESTING ASPECTS: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives wi...

Full description

Saved in:
Bibliographic Details
Published in:Frontiers in computational neuroscience Vol. 6; p. 97
Main Authors: Rückert, Elmar A., Neumann, Gerhard, Toussaint, Marc, Maass, Wolfgang
Format: Journal Article
Language:English
Published: Switzerland Frontiers Research Foundation 02.01.2013
Frontiers Media S.A
Subjects:
Artificial intelligence
Automation
Computer science
Distance learning
Dynamical systems
Graphical Models
International conferences
Machine learning
Motor ability
Motor planning
Motor skill
Motor skill learning
movement primitives
Neural networks
Neuroscience
optimal control
Principles
reinforcement learning
Robotics
Skills
Canada
Montreal Quebec Canada
optimal control
motor planning
graphical models
movement primitives
reinforcement learning
ISSN:1662-5188, 1662-5188
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:BIOLOGICAL MOVEMENT GENERATION COMBINES THREE INTERESTING ASPECTS: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative MP representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow MPs with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control (SOC) methods within a MP. The parameterization of the primitive is a graphical model that represents the dynamics and intrinsic cost function such that inference in this graphical model yields the control policy. We parameterize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning (RL) setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement representation facilitates learning significantly and leads to better generalization to new task settings without re-learning.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
Edited by: Andrea D'Avella, IRCCS Fondazione Santa Lucia, Italy
Reviewed by: Cees Van Leeuwen, Katholieke Universiteit Leuven, Belgium; Andrey Olypher, Emory University, USA; Petar Kormushev, Italian Institute of Technology, Italy
ISSN:1662-5188
1662-5188
DOI:10.3389/fncom.2012.00097