Nonsmooth Optimal Value and Policy Functions in Mechanical Systems Subject to Unilateral Constraints

State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator parameters. Unfortunately, we show that value and policy functions that arise in optimal control of mechanical systems subject to unilateral...

Full description

Saved in:
Bibliographic Details
Published in:IEEE control systems letters Vol. 4; no. 2; pp. 506 - 511
Main Authors: Banjanin, Bora S., Burden, Samuel A.
Format: Journal Article
Language:English
Published: IEEE 01.04.2020
Subjects:
Cost function
Force
Heuristic algorithms
hybrid systems
Mechanical systems
Optimal control
robotics
Robots
Trajectory
ISSN:2475-1456, 2475-1456
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator parameters. Unfortunately, we show that value and policy functions that arise in optimal control of mechanical systems subject to unilateral constraints - i.e., the contact-rich dynamics of robot locomotion and manipulation - are generally nonsmooth due to the underlying dynamics exhibiting discontinuous or piecewise-differentiable trajectory outcomes. Simple mechanical systems are used to illustrate this result and the implications for optimal control of contact-rich robot dynamics.
AbstractList State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator parameters. Unfortunately, we show that value and policy functions that arise in optimal control of mechanical systems subject to unilateral constraints - i.e., the contact-rich dynamics of robot locomotion and manipulation - are generally nonsmooth due to the underlying dynamics exhibiting discontinuous or piecewise-differentiable trajectory outcomes. Simple mechanical systems are used to illustrate this result and the implications for optimal control of contact-rich robot dynamics.
Author Burden, Samuel A.
Banjanin, Bora S.
Author_xml – sequence: 1
  givenname: Bora S.
  orcidid: 0000-0002-6493-7877
  surname: Banjanin
  fullname: Banjanin, Bora S.
  email: borab@uw.edu
  organization: Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
– sequence: 2
  givenname: Samuel A.
  orcidid: 0000-0002-2705-7395
  surname: Burden
  fullname: Burden, Samuel A.
  email: sburden@uw.edu
  organization: Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA
BookMark eNpNkM9KAzEQh4NUsNa-gF7yAq35v5ujLNYKqxXWCp6WbDZLU7ZJ2aSHvr2pLeJlZmB-38B8t2DkvDMA3GM0xxjJx7Kovqs5QVjOiRSIMXIFxoRlfIYZF6N_8w2YhrBFCOGcZIjIMWjfvQs77-MGrvbR7lQPv1R_MFC5Fn743uojXBycjjbloHXwzeiNclanYHUM0ewCrA7N1ugIo4drZ3sVzZC2RQLioKyL4Q5cd6oPZnrpE7BePH8Wy1m5enktnsqZJjiPqWrecdHppqMqk0JnROOuaUSGtJAKc2YQYobRHCuZCIJkJhpKZZs3hGtGJ4Cc7-rBhzCYrt4P6aXhWGNUn1TVv6rqk6r6oipBD2fIGmP-gFxSzpikP4eraDw
CODEN ICSLBO
Cites_doi 10.1145/3049797.3049807
10.1007/s002050000105
10.1177/0278364913506757
10.1007/BFb0121154
10.1287/moor.16.2.292
10.1109/ICRA.2017.7989259
10.1177/0278364909344635
10.1007/BF00992696
10.1126/scirobotics.aau5872
10.1287/moor.13.4.660
10.1177/027836499401300106
10.1109/LRA.2019.2900840
10.1023/A:1022921624832
10.1007/978-1-4614-4340-7
10.1145/2185520.2185539
10.1137/050625060
10.1177/027836498400300205
10.1109/TAC.2015.2404231
10.1109/TRO.2017.2783371
10.1613/jair.806
10.1177/0278364916639380
ContentType Journal Article
DBID 97E
RIA
RIE
AAYXX
CITATION
DOI 10.1109/LCSYS.2019.2960442
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISSN 2475-1456
EndPage 511
ExternalDocumentID 10_1109_LCSYS_2019_2960442
8935449
Genre orig-research
GrantInformation_xml – fundername: Army Research Office
  grantid: W911NF-16-1-0158
  funderid: 10.13039/100000183
– fundername: NSF Cyber-Physical Systems Award
  grantid: 1836819
GroupedDBID 0R~
6IK
97E
AAJGR
AASAJ
AAWTH
ABAZT
ABJNI
ABQJQ
ABVLG
ACGFS
AGQYO
AHBIQ
AKJIK
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
EJD
IFIPE
IPLJI
JAVBF
OCL
RIA
RIE
AAYXX
CITATION
ID FETCH-LOGICAL-c218t-c2c5f56fcbf3a796c72c1fbb670c69a154e004e4381a921820976b339d8b25c43
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000618810600020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2475-1456
IngestDate Sat Nov 29 06:10:50 EST 2025
Wed Aug 27 06:27:17 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 2
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c218t-c2c5f56fcbf3a796c72c1fbb670c69a154e004e4381a921820976b339d8b25c43
ORCID 0000-0002-2705-7395
0000-0002-6493-7877
PageCount 6
ParticipantIDs crossref_primary_10_1109_LCSYS_2019_2960442
ieee_primary_8935449
PublicationCentury 2000
PublicationDate 2020-April
2020-4-00
PublicationDateYYYYMMDD 2020-04-01
PublicationDate_xml – month: 04
  year: 2020
  text: 2020-April
PublicationDecade 2020
PublicationTitle IEEE control systems letters
PublicationTitleAbbrev LCSYS
PublicationYear 2020
Publisher IEEE
Publisher_xml – name: IEEE
References ref13
ref15
polak (ref4) 1997
ref14
(ref12) 0
ref1
ref17
schulman (ref10) 2015; 37
ref16
ref19
ref18
banjanin (ref28) 2019
ref24
ref23
ref26
bertsekas (ref5) 1996
ref20
ref22
heess (ref11) 0
ref21
ref27
ref29
ref8
ref7
ref9
ref3
ref6
brogliato (ref2) 1999
mayhew (ref25) 2010
References_xml – volume: 37
  start-page: 1889
  year: 2015
  ident: ref10
  article-title: Trust region policy optimization
  publication-title: Proc 32nd Int Conf Mach Learn
– year: 1999
  ident: ref2
  publication-title: Nonsmooth Mechanics Models Dynamics and Control
– ident: ref18
  doi: 10.1145/3049797.3049807
– ident: ref1
  doi: 10.1007/s002050000105
– ident: ref6
  doi: 10.1177/0278364913506757
– ident: ref16
  doi: 10.1007/BFb0121154
– year: 2019
  ident: ref28
  article-title: Data-driven modeling for hybrid dynamical systems
– ident: ref23
  doi: 10.1287/moor.16.2.292
– ident: ref19
  doi: 10.1109/ICRA.2017.7989259
– ident: ref20
  doi: 10.1177/0278364909344635
– ident: ref14
  doi: 10.1007/BF00992696
– year: 1997
  ident: ref4
  publication-title: Optimization Algorithms and Consistent Approximations
– year: 1996
  ident: ref5
  publication-title: Neuro-Dynamic Programming
– year: 2010
  ident: ref25
  article-title: Hybrid control for topologically constrained systems
– ident: ref13
  doi: 10.1126/scirobotics.aau5872
– ident: ref22
  doi: 10.1287/moor.13.4.660
– year: 0
  ident: ref12
  article-title: Learning dexterous in-hand manipulation
  publication-title: arXiv 1808 00177
– ident: ref21
  doi: 10.1177/027836499401300106
– ident: ref7
  doi: 10.1109/LRA.2019.2900840
– ident: ref17
  doi: 10.1023/A:1022921624832
– ident: ref15
  doi: 10.1007/978-1-4614-4340-7
– year: 0
  ident: ref11
  article-title: Emergence of locomotion behaviours in rich environments
  publication-title: arXiv 1707 02286
– ident: ref8
  doi: 10.1145/2185520.2185539
– ident: ref29
  doi: 10.1137/050625060
– ident: ref27
  doi: 10.1177/027836498400300205
– ident: ref24
  doi: 10.1109/TAC.2015.2404231
– ident: ref9
  doi: 10.1109/TRO.2017.2783371
– ident: ref26
  doi: 10.1613/jair.806
– ident: ref3
  doi: 10.1177/0278364916639380
SSID ssj0001827029
Score 2.1077573
Snippet State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator...
SourceID crossref
ieee
SourceType Index Database
Publisher
StartPage 506
SubjectTerms Cost function
Force
Heuristic algorithms
hybrid systems
Mechanical systems
Optimal control
robotics
Robots
Trajectory
Title Nonsmooth Optimal Value and Policy Functions in Mechanical Systems Subject to Unilateral Constraints
URI https://ieeexplore.ieee.org/document/8935449
Volume 4
WOSCitedRecordID wos000618810600020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 2475-1456
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001827029
  issn: 2475-1456
  databaseCode: RIE
  dateStart: 20170101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELXaioGFDxVE-ZIHNkibOIkdj6iiYoCCVEBlis6OI1VqE9Sm_H7OTlpAYmGJoshD9M7Ou7vcuyPkKgaex0hknvEl8yImYg8AHbkEAIzJkSSEcsMmxHicTKfyuUVutloYY4wrPjN9e-v-5WelXttU2QC5NY4i2SZtIXit1frOpyRWWSU3uhhfDh6Gk_eJLd6SfWZbkETsF_f8GKbiuGS0_7-3OCB7jc9Ib2sjH5KWKbokG-N-WZQINH3CY7_ABW8wXxsKRUbrbr90hKTl9hWdFfTRWJGvtQlt2pRT_GrYNAytSoq-5xysGnlO7QxPNzmiWh2R19Hdy_Dea0YmeBq5usKrjhH7XKs8BCG5FkwHuVJc-JpLQH_J4Kkwtq8XSNe8Hd0RFYYySxSLdRQek05RFuaE0EAAs-GrCdBD5BlDC0IGAQelsyRnqkeuN2CmH3VnjNRFFL5MHfSphT5toO-RrkVyu7IB8fTvx2dkl9m41lXInJNOtVybC7KjP6vZannpDP8FMu2u2A
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEF5qFfTigyrW5x68adpk89yjFEvFNgqtUk9hdrOBQptIm_r7nd2mVcGLlxDCHsK3s_lmJjPzEXLjQ5D5SGSWsjmzPBb6FgA6chEAKJUhSYTCiE2EcRyNx_ylRu42vTBKKVN8plr61vzLTwu51KmyNnKr73l8i2xr5ayqW-s7oxLp3iq-7oyxebvfGb4PdfkWbzE9hMRjv9jnh5yKYZPuwf_e45DsV14jvV9t8xGpqbxB0hgtZlYg1PQZD_4MF7zBdKko5CldzfulXaQtY1l0ktOB0m2-eldoNaic4ndDJ2JoWVD0Pqeg-5GnVKt4Gu2IcnFMXrsPo07PqkQTLIlsXeJV-oh-JkXmQsgDGTLpZEIEoS0DDugxKTwXSk_2Am7Gt6NDIlyXp5FgvvTcE1LPi1ydEuqEwHQAqxz0EYOU4R5CCk4AQqZRxkST3K7BTD5WszESE1PYPDHQJxr6pIK-SRoayc3KCsSzvx9fk93eaNBP-o_x0znZYzrKNfUyF6RezpfqkuzIz3KymF8ZI_gCe6qyIQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Nonsmooth+Optimal+Value+and+Policy+Functions+in+Mechanical+Systems+Subject+to+Unilateral+Constraints&rft.jtitle=IEEE+control+systems+letters&rft.au=Banjanin%2C+Bora+S.&rft.au=Burden%2C+Samuel+A.&rft.date=2020-04-01&rft.pub=IEEE&rft.eissn=2475-1456&rft.volume=4&rft.issue=2&rft.spage=506&rft.epage=511&rft_id=info:doi/10.1109%2FLCSYS.2019.2960442&rft.externalDocID=8935449
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1456&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1456&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1456&client=summon