Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis

Robotic prostheses deliver greater function than passive prostheses, but we face the challenge of tuning a large number of control parameters in order to personalize the device for individual amputee users. This problem is not easily solved by traditional control designs or the latest robotic techno...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on cybernetics Jg. 50; H. 6; S. 2346 - 2356
Hauptverfasser: Wen, Yue, Si, Jennie, Brandt, Andrea, Gao, Xiang, Huang, He Helen
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States IEEE 01.06.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Adult
Algorithms
Amputees - rehabilitation
Approximate dynamic programming (ADP)
Automation
Biomechanical Phenomena - physiology
direct heuristic dynamic programming (dHDP)
Dynamic programming
Exoskeleton Device
Feasibility
Gait
Gait - physiology
Humans
Impedance
Kinematics
Knee
Knee Prosthesis
Learning
Male
Noise measurement
Order parameters
Prostheses
Prosthetics
reinforcement learning (RL)
Reinforcement, Psychology
robotic knee prosthesis
Robotics
Robots
Signal Processing, Computer-Assisted
Treadmills
Tuning
Walking
Young Adult
ISSN:2168-2267, 2168-2275, 2168-2275
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Robotic prostheses deliver greater function than passive prostheses, but we face the challenge of tuning a large number of control parameters in order to personalize the device for individual amputee users. This problem is not easily solved by traditional control designs or the latest robotic technology. Reinforcement learning (RL) is naturally appealing. The recent, unprecedented success of AlphaZero demonstrated RL as a feasible, large-scale problem solver. However, the prosthesis-tuning problem is associated with several unaddressed issues such as that it does not have a known and stable model, the continuous states and controls of the problem may result in a curse of dimensionality, and the human-prosthesis system is constantly subject to measurement noise, environmental change and human-body-caused variations. In this paper, we demonstrated the feasibility of direct heuristic dynamic programming, an approximate dynamic programming (ADP) approach, to automatically tune the 12 robotic knee prosthesis parameters to meet individual human users' needs. We tested the ADP-tuner on two subjects (one able-bodied subject and one amputee subject) walking at a fixed speed on a treadmill. The ADP-tuner learned to reach target gait kinematics in an average of 300 gait cycles or 10 min of walking. We observed improved ADP tuning performance when we transferred a previously learned ADP controller to a new learning session with the same subject. To the best of our knowledge, our approach to personalize robotic prostheses is the first implementation of online ADP learning control to a clinical problem involving human subjects.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2168-2267
2168-2275
2168-2275
DOI:10.1109/TCYB.2019.2890974