Fourier-Hermite Dynamic Programming for Optimal Control

In this paper, we propose a novel computational method for solving non-linear optimal control problems. The method is based on the use of Fourier-Hermite series for approximating the action-value function arising in dynamic programming instead of the conventional Taylor-series expansion used in diff...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on automatic control Ročník 68; číslo 10; s. 1 - 8
Hlavní autori:	Hassan, Syeda Sakira, Sarkka, Simo
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.10.2023 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	approximate dynamic programming Convergence Costs differential dynamic programming Dynamic programming Fourier series Fourier–Hermite series Heuristic algorithms Jacobian matrices Nonlinear control Optimal control Series expansion sigma-point dynamic programming Taylor series trajectory optimization
ISSN:	0018-9286, 1558-2523
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	In this paper, we propose a novel computational method for solving non-linear optimal control problems. The method is based on the use of Fourier-Hermite series for approximating the action-value function arising in dynamic programming instead of the conventional Taylor-series expansion used in differential dynamic programming (DDP). The coefficients of the Fourier-Hermite series can be numerically computed by using sigma-point methods, which leads to a novel class of sigma-point based dynamic programming methods. We also prove the quadratic convergence of the method and experimentally test its performance against other methods.
AbstractList	In this paper, we propose a novel computational method for solving non-linear optimal control problems. The method is based on the use of Fourier-Hermite series for approximating the action-value function arising in dynamic programming instead of the conventional Taylor-series expansion used in differential dynamic programming (DDP). The coefficients of the Fourier-Hermite series can be numerically computed by using sigma-point methods, which leads to a novel class of sigma-point based dynamic programming methods. We also prove the quadratic convergence of the method and experimentally test its performance against other methods. In this article, we propose a novel computational method for solving nonlinear optimal control problems. The method is based on the use of Fourier–Hermite series for approximating the action-value function arising in dynamic programming instead of the conventional Taylor-series expansion used in differential dynamic programming. The coefficients of the Fourier–Hermite series can be numerically computed by using sigma-point methods, which leads to a novel class of sigma-point-based dynamic programming methods. We also prove the quadratic convergence of the method and experimentally test its performance against other methods.
Author	Hassan, Syeda Sakira Sarkka, Simo
Author_xml	– sequence: 1 givenname: Syeda Sakira orcidid: 0000-0001-6838-474X surname: Hassan fullname: Hassan, Syeda Sakira organization: Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland – sequence: 2 givenname: Simo orcidid: 0000-0002-7031-9354 surname: Sarkka fullname: Sarkka, Simo organization: Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland
BookMark	eNp9kL1vwjAQxa2KSgXavUOHSJ1D7VzijxHRUioh0YHOlp1ckFESUycM_Pc1gqHq0Ol00vvdvfcmZNT5Dgl5ZHTGGFUv2_liltEMZpBBngG_IWNWFDLNigxGZEwpk6nKJL8jk77fx5XnORsTsfTH4DCkKwytGzB5PXWmdWXyGfwumLZ13S6pfUg2h8G1pkkWvhuCb-7JbW2aHh-uc0q-lm_bxSpdb94_FvN1WgLkQyo5twKRUSGEMTa3tmZKlVxaU4MUtkJqgIHhVYnCKslsaaipojtZIeYCpuT5cvcQ_PcR-0Hvo-EuvtQxjAJQvICo4hdVGXzfB6x16QYzuLNX4xrNqD6XpGNJ-lySvpYUQfoHPIQYM5z-Q54uiEPEX3JKC6E4_AAoXHPU
CODEN	IETAA9
CitedBy_id	crossref_primary_10_1016_j_comnet_2025_111592 crossref_primary_10_1016_j_asoc_2025_113391 crossref_primary_10_1016_j_est_2025_116807 crossref_primary_10_1016_j_ifacol_2025_08_089 crossref_primary_10_1016_j_isatra_2024_11_013 crossref_primary_10_1016_j_sigpro_2024_109547
Cites_doi	10.1109/CDC.2016.7798817 10.1017/CBO9780511546877 10.1007/BF02162032 10.1109/ADPRL.2011.5967350 10.2307/2002797 10.1137/1.9780898717761 10.1109/TAC.2020.3025870 10.1109/IROS.2016.7759229 10.1109/9.855552 10.1109/TSP.2006.875389 10.1016/0096-3003(84)90051-1 10.1016/j.ifacol.2019.11.062 10.1109/LRA.2021.3098928 10.1109/TAC.2009.2017966 10.1109/TAC.2009.2019800 10.1177/1687814018782330 10.1016/B978-0-12-012710-8.50010-8 10.1109/9.86943 10.1109/TSMC.1983.6313077 10.1109/IROS.2012.6386025 10.1017/CBO9781139344203 10.1109/TAC.2011.2174667 10.1080/00207176608921369 10.1109/9.847726 10.1007/BF00934463
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID	97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D
DOI	10.1109/TAC.2023.3234236
DatabaseName	IEEE Xplore (IEEE) IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Xplore CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-2523
EndPage	8
ExternalDocumentID	10_1109_TAC_2023_3234236 10005796
Genre	orig-research
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS ESBDL F5P HZ~ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 ~02 3EH 5VS AAYXX AETIX AGSQL AI. AIBXA ALLEH CITATION EJD H~9 IAAWW IBMZZ ICLAB IDIHD IFJZH VH1 VJK 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c334t-866b7ee10777aab4bbf199c68baf387bde0a313a6dce7b981bca0ad0168dee473
IEDL.DBID	RIE
ISICitedReferencesCount	5
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001114541100005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9286
IngestDate	Mon Jun 30 10:12:56 EDT 2025 Sat Nov 29 05:41:07 EST 2025 Tue Nov 18 22:20:17 EST 2025 Wed Aug 27 02:14:01 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	10
Language	English
License	https://creativecommons.org/licenses/by/4.0/legalcode
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c334t-866b7ee10777aab4bbf199c68baf387bde0a313a6dce7b981bca0ad0168dee473
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-7031-9354 0000-0001-6838-474X
OpenAccessLink	https://ieeexplore.ieee.org/document/10005796
PQID	2869339653
PQPubID	85475
PageCount	8
ParticipantIDs	proquest_journals_2869339653 ieee_primary_10005796 crossref_citationtrail_10_1109_TAC_2023_3234236 crossref_primary_10_1109_TAC_2023_3234236
PublicationCentury	2000
PublicationDate	2023-10-01
PublicationDateYYYYMMDD	2023-10-01
PublicationDate_xml	– month: 10 year: 2023 text: 2023-10-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on automatic control
PublicationTitleAbbrev	TAC
PublicationYear	2023
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref35 tassa (ref27) 2011 ref14 bertsekas (ref2) 2000; i haykin (ref36) 2009; 54 abadi (ref31) 0 ref30 ref11 mayne (ref13) 1973; 10 ref39 ref16 bellman (ref10) 1957 ref19 (ref33) 0 ref18 abbeel (ref6) 2006; 19 bertsekas (ref3) 2011; ii särkkä (ref37) 2015; 11 kokkala (ref42) 2016; 11 ref23 liao (ref15) 1990 ref26 ref25 ref20 bu?oniu (ref9) 2017 sutton (ref1) 2018 ref41 malliavin (ref34) 2015 ref22 ref44 ref21 ref43 camacho (ref5) 2013 ref28 ref29 ref8 paszke (ref32) 0 li (ref17) 0 ref7 ref4 jacobson (ref12) 1970 särkkä (ref38) 0 ref40 pan (ref24) 2014; 27
References_xml	– ident: ref18 doi: 10.1109/CDC.2016.7798817 – ident: ref4 doi: 10.1017/CBO9780511546877 – year: 2013 ident: ref5 publication-title: Model Predictive Control – ident: ref41 doi: 10.1007/BF02162032 – year: 2011 ident: ref27 article-title: Theory and implementation of biomimetic motor controllers – year: 2018 ident: ref1 publication-title: Reinforcement Learning An Introduction – ident: ref23 doi: 10.1109/ADPRL.2011.5967350 – ident: ref26 doi: 10.2307/2002797 – ident: ref30 doi: 10.1137/1.9780898717761 – ident: ref8 doi: 10.1109/TAC.2020.3025870 – ident: ref20 doi: 10.1109/IROS.2016.7759229 – ident: ref35 doi: 10.1109/9.855552 – start-page: 222 year: 0 ident: ref17 article-title: Iterative linear quadratic regulator design for nonlinear biological movement systems publication-title: Proc IEEE 1st Int Conf Inform Control Autom Robot – ident: ref39 doi: 10.1109/TSP.2006.875389 – start-page: 265 year: 0 ident: ref31 article-title: TensorFlow: A system for large-scale machine learning publication-title: Proc 12th USENIX Conf Oper Syst Des Implementation – ident: ref29 doi: 10.1016/0096-3003(84)90051-1 – year: 1957 ident: ref10 publication-title: Dynamic Programming – start-page: 1 year: 0 ident: ref38 article-title: Gaussian process quadratures in nonlinear sigma-point filtering and smoothing publication-title: Proc 17th Int Conf Inf Fusion – ident: ref19 doi: 10.1016/j.ifacol.2019.11.062 – volume: i year: 2000 ident: ref2 publication-title: Dynamic Programming and Optimal Control – ident: ref40 doi: 10.1109/LRA.2021.3098928 – year: 2015 ident: ref34 publication-title: Stochastic Analysis – volume: 11 start-page: 31 year: 2015 ident: ref37 article-title: On the relation between gaussian process quadratures and sigma-point methods publication-title: J Adv Inf Fusion – year: 0 ident: ref32 article-title: Automatic differentiation in PyTorch publication-title: Proc NeurIPS Autodiff Workshop Future Gradient Mach Learn Softw Techn – year: 1970 ident: ref12 publication-title: Differential Dynamic Programming – ident: ref7 doi: 10.1109/TAC.2009.2017966 – volume: 54 start-page: 1254 year: 2009 ident: ref36 article-title: Cubature Kalman filters publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.2009.2019800 – ident: ref44 doi: 10.1177/1687814018782330 – year: 1990 ident: ref15 article-title: The proof of quadratic convergence of differential dynamic programming – year: 0 ident: ref33 article-title: Deep Learning Toolbox, 2021, the MathWorks – volume: 10 start-page: 179 year: 1973 ident: ref13 article-title: Differential dynamic programming-a unified approach to the optimization of dynamic systems publication-title: Control and Dynamic Systems doi: 10.1016/B978-0-12-012710-8.50010-8 – ident: ref16 doi: 10.1109/9.86943 – ident: ref43 doi: 10.1109/TSMC.1983.6313077 – ident: ref28 doi: 10.1109/IROS.2012.6386025 – year: 2017 ident: ref9 publication-title: Reinforcement Learning and Dynamic Programming Using Function Approximators – ident: ref22 doi: 10.1017/CBO9781139344203 – volume: ii year: 2011 ident: ref3 publication-title: Dynamic Programming and Optimal Control – volume: 11 start-page: 15 year: 2016 ident: ref42 article-title: Sigma-point filtering and smoothing based parameter estimation in nonlinear dynamic systems publication-title: J Adv Inf Fusion – volume: 19 start-page: 1 year: 2006 ident: ref6 article-title: An application of reinforcement learning to aerobatic helicopter flight publication-title: Proc Adv Neural Inf Process Syst – ident: ref25 doi: 10.1109/TAC.2011.2174667 – ident: ref11 doi: 10.1080/00207176608921369 – ident: ref21 doi: 10.1109/9.847726 – volume: 27 start-page: 1907 year: 2014 ident: ref24 article-title: Probabilistic differential dynamic programming publication-title: Proc Adv Neural Inf Process Syst – ident: ref14 doi: 10.1007/BF00934463
SSID	ssj0016441
Score	2.4654303
Snippet	In this paper, we propose a novel computational method for solving non-linear optimal control problems. The method is based on the use of Fourier-Hermite... In this article, we propose a novel computational method for solving nonlinear optimal control problems. The method is based on the use of Fourier–Hermite...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1
SubjectTerms	approximate dynamic programming Convergence Costs differential dynamic programming Dynamic programming Fourier series Fourier–Hermite series Heuristic algorithms Jacobian matrices Nonlinear control Optimal control Series expansion sigma-point dynamic programming Taylor series trajectory optimization
Title	Fourier-Hermite Dynamic Programming for Optimal Control
URI	https://ieeexplore.ieee.org/document/10005796 https://www.proquest.com/docview/2869339653
Volume	68
WOSCitedRecordID	wos001114541100005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2523 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016441 issn: 0018-9286 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07TwJBEJ4IsdDCJ0YUzRU2Fgd37LGPkqCEwiAFGrrLPobERMDA4e93HwfBGE3srpjdXL7ZnZndnZkP4G5KEs1EamLGTRZnU0NjKRSNTWaI6biW5Z4b8PWJDYd8MhGjsljd18Igok8-w6b79G_5ZqHX7qqs5e6iXe1kBSqMsVCstX0ycI49mF27g9t8-yaZiNa422s6mvAmabuGd_SbD_KkKj8ssXcv_eN__tgJHJVxZNQNij-FPZyfweFOd8FzYP3ARxcPXMJLgdFDYJ-PRiEna2alIhuzRs_WbMzsZL2Qtl6Dl_7juDeIS56EWBOSFTGnVDFEe5BjTEqVKTVNhdCUKzklnCmDiSQpkdRoZErYQFXLRBoLGTeIGSMXUJ0v5ngJkY3_dMZkO5XoGs0RYeMVVIbboEooTlUdWhvkcl02EXdcFu-5P0wkIrdY5w7rvMS6DvfbER-hgcYfsjWH7Y5cgLUOjY128nKLrXKrYkGIoB1y9cuwazhws4fUuwZUi-Uab2BffxZvq-WtXz1fgwDAcA
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEJ4omqgHnxhR1D148bCwuy19HAlKMCJyQMNt09cmJgKGh7_fdrsQjNHE2x6mu5tv2um08_gAbjIUKcpjHVKmcYgzTULBJQk11kg3XMvynBvwtUt7PTYc8n5RrJ7Xwhhj8uQzU3OPeSxfT9TCXZXV3V20q53chK0Gxknsy7VWQQO3tXvDa9dwwlZRyYjXB81WzRGF11DiWt6Rb7tQTqvywxbnG0z74J-_dgj7hScZNL3qj2DDjI9hb62_4AnQtmekCzsu5WVugjvPPx_0fVbWyEoF1msNnq3hGNmXtXziehle2veDVicsmBJChRCeh4wQSY2xRzlKhZBYyizmXBEmRYYYldpEAsVIEK0Mldy6qkpEQlvImDYGU3QKpfFkbM4gsB6gwlQksTCu1Rzi1mMxUjPrVnHJiKxAfYlcqoo24o7N4j3NjxMRTy3WqcM6LbCuwO1qxIdvofGHbNlhuybnYa1AdamdtFhks9SqmCPESQOd_zLsGnY6g6du2n3oPV7ArvuST8SrQmk-XZhL2Faf87fZ9CqfSV-zzsO3
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Fourier-Hermite+Dynamic+Programming+for+Optimal+Control&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Hassan%2C+Syeda+Sakira&rft.au=Sarkka%2C+Simo&rft.date=2023-10-01&rft.pub=IEEE&rft.issn=0018-9286&rft.spage=1&rft.epage=8&rft_id=info:doi/10.1109%2FTAC.2023.3234236&rft.externalDocID=10005796
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon