Optimal balancing control of bipedal robots using reinforcement learning

The balance control a bipedal robot in the presence of external disturbances is still a challenge. In this paper, a novel optimal ankle stiffness regulation for humanoid robot balancing problem is proposed. The presented techniques are developed based on the integral reinforcement learning (IRL) alg...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2016 12th World Congress on Intelligent Control and Automation (WCICA) S. 2186 - 2191
Hauptverfasser: Fang Peng, Lijia Ding, Zhijun Li, Chenguang Yang, Chun-Yi Su
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.06.2016
Schlagworte:
Cost function
Impedance
Mathematical model
Riccati equations
Robot kinematics
Torque
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract The balance control a bipedal robot in the presence of external disturbances is still a challenge. In this paper, a novel optimal ankle stiffness regulation for humanoid robot balancing problem is proposed. The presented techniques are developed based on the integral reinforcement learning (IRL) algorithm, which is designed for unknown continuous-time systems using only partial knowledge of the system dynamics. A linear mathematical model of an inverted pendulum model (IPM) is employed to study robot balance, and optimization is achieved by using linear quadratic regulator (LQR). Because the internal system dynamics is nonlinear and time-varying, IRL algorithm is proposed to solve the algebraic Riccati equation online without knowledge of the internal system dynamics. Nonlinear stiffness stabilizer based on both IRL and the fixed stiffness are studied in the simulation, the comparative results demonstrate the superior balance ability of the proposed method. In addition, dynamics changing is also discussed in the simulation to test robustness of the proposed method.
AbstractList The balance control a bipedal robot in the presence of external disturbances is still a challenge. In this paper, a novel optimal ankle stiffness regulation for humanoid robot balancing problem is proposed. The presented techniques are developed based on the integral reinforcement learning (IRL) algorithm, which is designed for unknown continuous-time systems using only partial knowledge of the system dynamics. A linear mathematical model of an inverted pendulum model (IPM) is employed to study robot balance, and optimization is achieved by using linear quadratic regulator (LQR). Because the internal system dynamics is nonlinear and time-varying, IRL algorithm is proposed to solve the algebraic Riccati equation online without knowledge of the internal system dynamics. Nonlinear stiffness stabilizer based on both IRL and the fixed stiffness are studied in the simulation, the comparative results demonstrate the superior balance ability of the proposed method. In addition, dynamics changing is also discussed in the simulation to test robustness of the proposed method.
Author Zhijun Li
Lijia Ding
Fang Peng
Chun-Yi Su
Chenguang Yang
Author_xml – sequence: 1
  surname: Fang Peng
  fullname: Fang Peng
  organization: Sch. of Autom. Sci. & Eng., South China Univ. of Technol., Guangzhou, China
– sequence: 2
  surname: Lijia Ding
  fullname: Lijia Ding
  organization: Sch. of Autom. Sci. & Eng., South China Univ. of Technol., Guangzhou, China
– sequence: 3
  surname: Zhijun Li
  fullname: Zhijun Li
  email: zjli@ieee.org
  organization: Sch. of Autom. Sci. & Eng., South China Univ. of Technol., Guangzhou, China
– sequence: 4
  surname: Chenguang Yang
  fullname: Chenguang Yang
  email: cyang@ieee.org
  organization: Sch. of Autom. Sci. & Eng., South China Univ. of Technol., Guangzhou, China
– sequence: 5
  surname: Chun-Yi Su
  fullname: Chun-Yi Su
  organization: Sch. of Autom. Sci. & Eng., South China Univ. of Technol., Guangzhou, China
BookMark eNotj8tKBDEURCPowhn9Ad3kB7rNTcxrKY06wsAsFFwOedxIoCdp0u3Cv7fFWRXUgeLUhlyWWpCQO2A9ALMPn8Pb8NRzBqrXUptHJS7IBiSzjAtp5DXZHaYln9xIvRtdCbl80VDL0upIa6I-TxhX2Kqvy0y_5z_eMJdUW8ATloWO6FpZ6xtyldw44-05t-T95flj2HX7w-vqsO-yZUsXORdglA46xMhj8iH5GCAGrRlHZUEYZZnzLIJi0hgHDhMKZYP23IPYkvv_1YyIx6mt6u3neL4mfgE9P0n4
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/WCICA.2016.7578463
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 1509023585
9781467384148
9781509023585
1467384143
EndPage 2191
ExternalDocumentID 7578463
Genre orig-research
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i90t-d2231867c7cdd2dfbcfbdc1dc7702e69138690ab0d160588a1aefe369c7b2b13
IEDL.DBID RIE
IngestDate Thu Jun 29 18:37:48 EDT 2023
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i90t-d2231867c7cdd2dfbcfbdc1dc7702e69138690ab0d160588a1aefe369c7b2b13
PageCount 6
ParticipantIDs ieee_primary_7578463
PublicationCentury 2000
PublicationDate 2016-June
PublicationDateYYYYMMDD 2016-06-01
PublicationDate_xml – month: 06
  year: 2016
  text: 2016-June
PublicationDecade 2010
PublicationTitle 2016 12th World Congress on Intelligent Control and Automation (WCICA)
PublicationTitleAbbrev WCICA
PublicationYear 2016
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.6096089
Snippet The balance control a bipedal robot in the presence of external disturbances is still a challenge. In this paper, a novel optimal ankle stiffness regulation...
SourceID ieee
SourceType Publisher
StartPage 2186
SubjectTerms Cost function
Impedance
Mathematical model
Riccati equations
Robot kinematics
Torque
Title Optimal balancing control of bipedal robots using reinforcement learning
URI https://ieeexplore.ieee.org/document/7578463
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NSwMxEB3a4sGTSitqVXLw6Lb71U1ylGKpILWgaG8lH5NS0G7Zbv397qRrRfDiLWQCIQlhmJn35gHcEFhNUFVXaCmDVLk0UNKoAK1wRPw0sVdreH3kk4mYzeS0Abd7LgwievAZ9mjoa_k2N1tKlfWp93qaJU1ocp7tuFrfPJhQ9t-GD8M7AmtlvXrhL8UU7zBGR__b6hg6P8w7Nt37lBNo4KoN46fqX3-od6YJh2gqA6sR5ix3TC_XaCtjkeu83DACsi9Ygb4jqvHJP1ZLQyw68Dy6fxmOg1oBIVjKsAxs5bup4ZzhxtrYOm2ctiayhvMwxkxGCelJKR3aiKqbQkUKHSaZNFzHOkpOobXKV3gGLK3CLGsGA41VSKSiKkZIXSzQZdom2gk8hzbdwXy9a3Exr49_8fd0Fw7pmneIqUtolcUWr-DAfJbLTXHt3-UL5UKSoQ
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5qFfSk0opvc_Dotvvq7uYoxdJirQWL9lbymJQF7Zbt1t9vZrtWBC_eQiYQkhCGmfm--QBuCayWUFU3kZw7oTChI7gSDurEEPFT-aVaw-swHo2S6ZSPa3C35cIgYgk-wxYNy1q-ztSaUmVt6r0eRsEO7JJyVsXW-mbCuLz91h107wmuFbWqpb80U0qX0Tv832ZH0Pzh3rHx1qscQw0XDeg_25_9Id6ZJCSisgZWYcxZZphMl6itMc9kVqwYQdnnLMeyJ6oq03-sEoeYN-Gl9zDp9p1KA8FJuVs42npvajmnYqW1r41URmrlaRXHro8R9wJSlBLS1R7VNxPhCTQYRFzF0pdecAL1RbbAU2ChDbS06nQk2qBIeDZKCI2foImkDqRJ8AwadAez5abJxaw6_vnf0zew3588DWfDwejxAg7oyjf4qUuoF_kar2BPfRbpKr8u3-gLtl-V6g
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2016+12th+World+Congress+on+Intelligent+Control+and+Automation+%28WCICA%29&rft.atitle=Optimal+balancing+control+of+bipedal+robots+using+reinforcement+learning&rft.au=Fang+Peng&rft.au=Lijia+Ding&rft.au=Zhijun+Li&rft.au=Chenguang+Yang&rft.date=2016-06-01&rft.pub=IEEE&rft.spage=2186&rft.epage=2191&rft_id=info:doi/10.1109%2FWCICA.2016.7578463&rft.externalDocID=7578463