Transferring human emotions to robot motions using Neural Policy Style Transfer

Neural Style Transfer (NST) was originally proposed to use feature extraction capabilities of Neural Networks as a way to perform Style Transfer with images. Pre-trained image classification architectures were selected for feature extraction, leading to new images showing the same content as the ori...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Cognitive systems research Ročník 82; s. 101121
Hlavní autoři: Fernandez-Fernandez, Raul, Łukawski, Bartek, Victores, Juan G., Pacchierotti, Claudio
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier B.V 01.12.2023
Elsevier
Témata:
Autoencoders
Computer Science
Deep reinforcement learning
NPST
Robotics
Style transfer
TD3
Style transfer
Deep reinforcement learning
TD3
Autoencoders
NPST
ISSN:1389-0417, 1389-0417
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Neural Style Transfer (NST) was originally proposed to use feature extraction capabilities of Neural Networks as a way to perform Style Transfer with images. Pre-trained image classification architectures were selected for feature extraction, leading to new images showing the same content as the original but with a different style. In robotics, Style Transfer can be employed to transfer human motion styles to robot motions. The challenge lies in the lack of pre-trained classification architectures for robot motions that could be used for feature extraction. Neural Policy Style Transfer TD3 (NPST3) is proposed for the transfer of human motion styles to robot motions. This framework allows the same robot motion to be executed in different human-centered motion styles, such as in an “angry”, “happy”, “calm”, or “sad” fashion. The Twin Delayed Deep Deterministic Policy Gradient (TD3) network is introduced for the generation of control policies. An autoencoder network is in charge of feature extraction for the Style Transfer step. The Style Transfer step can be performed both offline and online: offline for the autonomous executions of human-style robot motions, and online for adapting at runtime the style of e.g., a teleoperated robot. The framework is tested using two different robotic platforms: a robotic manipulator designed for telemanipulation tasks, and a humanoid robot designed for social interaction. The proposed approach was evaluated for both platforms, performing a total of 147 questionnaires asking human subjects to recognize the human motion style transferred to the robot motion for a predefined set of actions. •Proposes the NPST3 framework for performing Style Transfer with robot motions.•Allows Style Transfer to be performed offline or online.•Introduces TD3 within an Style Transfer framework.•Experiments performed with two different robotic platforms and 147 questionnaires.•Extends the idea of Neural Style Transfer to robot motions.
ISSN:1389-0417
1389-0417
DOI:10.1016/j.cogsys.2023.05.010