Cooperative control of velocity and heading for unmanned surface vessel based on twin delayed deep deterministic policy gradient with an integral compensator
This paper addresses cooperative control of velocity and heading for an unmanned surface vessel (USV) utilizing a twin delay deep deterministic policy gradient (TD3) reinforcement learning algorithm. The utilization of a deep neural network establishes a direct correlation between the USV’s state pa...
Gespeichert in:
| Veröffentlicht in: | Ocean engineering Jg. 288; S. 115943 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Elsevier Ltd
15.11.2023
|
| Schlagworte: | |
| ISSN: | 0029-8018, 1873-5258 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | This paper addresses cooperative control of velocity and heading for an unmanned surface vessel (USV) utilizing a twin delay deep deterministic policy gradient (TD3) reinforcement learning algorithm. The utilization of a deep neural network establishes a direct correlation between the USV’s state parameters and motor control quantities. A reward function is devised to update the network parameters and which acquires the trained model. The introducing of an integral compensator effectively eliminates the steady-state error of the system, thereby significantly enhancing the precision of both velocity control and heading control. Furthermore, a two-stage training algorithm comprising offline learning and online learning has been devised. Through offline learning, a deep neural network model for the USV controller is obtained. Subsequently, the optimization of the controller strategy is conducted during the online learning phase. Ultimately, the simulation results demonstrate the exceptional control performance attained by the proposed algorithm.
•A twin delayed deep deterministic policy gradient algorithm with integral compensation (TD3-IC) is proposed.•A two-stage training algorithm is used to first train offline in a simulated environment, and then train online to optimize the control strategy.•The performance of TD3-IC controller is compared with other controllers in the experiment.•The generalization and anti-interference experiment of the model is carried out. |
|---|---|
| AbstractList | This paper addresses cooperative control of velocity and heading for an unmanned surface vessel (USV) utilizing a twin delay deep deterministic policy gradient (TD3) reinforcement learning algorithm. The utilization of a deep neural network establishes a direct correlation between the USV’s state parameters and motor control quantities. A reward function is devised to update the network parameters and which acquires the trained model. The introducing of an integral compensator effectively eliminates the steady-state error of the system, thereby significantly enhancing the precision of both velocity control and heading control. Furthermore, a two-stage training algorithm comprising offline learning and online learning has been devised. Through offline learning, a deep neural network model for the USV controller is obtained. Subsequently, the optimization of the controller strategy is conducted during the online learning phase. Ultimately, the simulation results demonstrate the exceptional control performance attained by the proposed algorithm.
•A twin delayed deep deterministic policy gradient algorithm with integral compensation (TD3-IC) is proposed.•A two-stage training algorithm is used to first train offline in a simulated environment, and then train online to optimize the control strategy.•The performance of TD3-IC controller is compared with other controllers in the experiment.•The generalization and anti-interference experiment of the model is carried out. |
| ArticleNumber | 115943 |
| Author | Zhao, Shulong Wang, Yibai Wang, Qingling |
| Author_xml | – sequence: 1 givenname: Yibai orcidid: 0009-0001-5467-1702 surname: Wang fullname: Wang, Yibai organization: School of Automation, Southeast University, Nanjing, 210096, Jiangsu, China – sequence: 2 givenname: Shulong surname: Zhao fullname: Zhao, Shulong organization: National University of Defense Technology, Changsha, Hunan, China – sequence: 3 givenname: Qingling surname: Wang fullname: Wang, Qingling email: csuwql@gmail.com organization: School of Automation, Southeast University, Nanjing, 210096, Jiangsu, China |
| BookMark | eNqFkM9qHDEMxk1JoZu0r1D8ArOxxvMXemhZmjSwkEt7Nh5Z3niZtQfb2bAPk3eNl20vveQiIaHvk_S7Zlc-eGLsK4g1COhu9-uApD353boWtVwDtGMjP7AVDL2s2rodrthKiHqsBgHDJ3ad0l4I0XVCrtjrJoSFos7uSByDzzHMPFh-pDmgyyeuveFPpI3zO25D5M_-oL0nw9NztBqpTKZEM590Ks3geX5xnhua9anUhmgpIVM8OO9SdsiXMDs88V0snuQzf3H5qWzhzmcqzblccVjIJ51D_Mw-Wj0n-vI337A_dz9_b35V28f7h82PbYUS6lwN49Q0fQstymGqrYBJWtMYNA2MvUACnAga6k2HRnZ1D2aACVptcbBjZ1DesG8XX4whpUhWld8LkzMP7WYFQp1Rq736h1qdUasL6iLv_pMv0R10PL0v_H4RUnnu6CiqhAUKknGRMCsT3HsWbzjNpGQ |
| CitedBy_id | crossref_primary_10_1016_j_oceaneng_2024_120175 crossref_primary_10_1080_17445302_2024_2391788 crossref_primary_10_1007_s11071_025_11636_9 crossref_primary_10_3390_jmse12050836 crossref_primary_10_1002_rnc_7620 crossref_primary_10_1002_rnc_7363 |
| Cites_doi | 10.1007/s10489-020-01819-9 10.1088/1755-1315/632/3/032046 10.1016/j.automatica.2019.108661 10.1017/S0373463320000284 10.1016/j.neucom.2017.06.042 10.1016/j.conengprac.2018.02.013 10.1109/TITS.2020.2989352 10.1016/j.oceaneng.2023.113670 10.1109/MCS.2015.2495095 10.1109/TNNLS.2020.3009214 10.1109/TII.2022.3142323 10.1109/TNNLS.2021.3068762 10.1016/j.oceaneng.2021.110477 10.1016/j.oceaneng.2019.04.099 |
| ContentType | Journal Article |
| Copyright | 2023 Elsevier Ltd |
| Copyright_xml | – notice: 2023 Elsevier Ltd |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.oceaneng.2023.115943 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Oceanography |
| EISSN | 1873-5258 |
| ExternalDocumentID | 10_1016_j_oceaneng_2023_115943 S0029801823023272 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 123 1B1 1~. 1~5 4.4 457 4G. 5VS 7-5 71M 8P~ 9JM 9JN AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAXUO ABFYP ABJNI ABLST ABMAC ABYKQ ACDAQ ACGFS ACRLP ADBBV ADEZE ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AFXIZ AGHFR AGUBO AGYEJ AHEUO AHHHB AHJVU AIEXJ AIKHN AITUG AJOXV AKIFW ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AXJTR BJAXD BKOJK BLECG BLXMC CS3 DU5 EBS EFJIC EFLBG EO8 EO9 EP2 EP3 FDB FIRID FNPLU FYGXN G-Q GBLVA IHE J1W JJJVA KCYFY KOM LY6 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 ROL RPZ SDF SDG SES SEW SPC SPCBC SSJ SST SSZ T5K TAE TN5 XPP ZMT ~02 ~G- 29N 6TJ 9DU AAQXK AATTM AAXKI AAYWO AAYXX ABFNM ABWVN ABXDB ACKIV ACLOT ACNNM ACRPL ACVFH ADCNI ADMUD ADNMO AEIPS AEUPX AFFNX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP ASPBG AVWKF AZFZN CITATION EFKBS EJD FEDTE FGOYB G-2 HVGLF HZ~ R2- SAC SET WUQ ~HD |
| ID | FETCH-LOGICAL-c312t-89b447515c38b2f01b3fd4dcd41970ce1cbe14e7d6cd36271d81b15afc8f96dc3 |
| ISICitedReferencesCount | 7 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001097846700001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0029-8018 |
| IngestDate | Sat Nov 29 07:30:50 EST 2025 Tue Nov 18 21:14:48 EST 2025 Fri Feb 23 02:34:03 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Twin delay deep deterministic policy gradient Neural network Reinforcement learning Unmanned surface vessel |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c312t-89b447515c38b2f01b3fd4dcd41970ce1cbe14e7d6cd36271d81b15afc8f96dc3 |
| ORCID | 0009-0001-5467-1702 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_oceaneng_2023_115943 crossref_primary_10_1016_j_oceaneng_2023_115943 elsevier_sciencedirect_doi_10_1016_j_oceaneng_2023_115943 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-11-15 |
| PublicationDateYYYYMMDD | 2023-11-15 |
| PublicationDate_xml | – month: 11 year: 2023 text: 2023-11-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationTitle | Ocean engineering |
| PublicationYear | 2023 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Kien, Anh, Son (b8) 2021; 51 Du, Lin, Zhang, Dong, Zhang (b4) 2022; 246 Fujimoto, Hoof, Meger (b6) 2018 Silver, D., Lever, G., Heess, N., Degris, T., Riedmiller, M., 2014. Deterministic Policy Gradient Algorithms. In: International Conference on Machine Learning. Fu, Wang (b5) 2022 Riani, Madani, Benallegue, Djouani (b16) 2018; 75 Xu, Pan, Huang, Zhang (b28) 2020; 73 Zhao, Qi, Ma, Li, Malekian, Sotelo (b33) 2020; 22 Zhang, Huang, Qin, Wan, Li, Cao, Su (b30) 2018; 151 Wang, Zhang, Zheng, Xiao, Yang (b24) 2016; 3 Woo, Yu, Kim (b25) 2019; 183 Zhang, G., Zeng, Q., Zhu, Z., Dai, X., Zhu, C., 2018a. Research on underwater safety inspection and operational robot motion control. In: 2018 33rd Youth Academic Annual Conference of Chinese Association of Automation (YAC). Wang, Gao, Zhao, Ahn (b21) 2020; 32 Wang, Liu, Tian, Zhang, Qiao, Wang (b22) 2023; 270 Lillicrap, Hunt, Pritzel, Heess, Erez, Tassa, Silver, Wierstra (b10) 2015 Sutton, Barto (b19) 1998 Wang, Du, Tao (b20) 2022; 248 Yi (b29) 2020; 112 Zhao, Ma, Hu (b32) 2021; 32 Xu, Lu, Liu, Cai, Zhang (b26) 2022 Chen, Lei, Li (b1) 2022 Li, Chen, Zou, Li (b9) 2017; 267 Niu, Wan (b14) 2021; 632 Cui, Peng, Li (b2) 2022; 18 Gonzalez-Garcia, Castañeda, Garrido (b7) 2020 Xu, Lu, Liu, Zhang (b27) 2020; 217 Wang, Shen, Sha, Li, Kong, Chen, He (b23) 2019 Liu, Wang, Tian (b11) 2022 Mnih, Kavukcuoglu, Silver, Graves, Antonoglou, Wierstra, Riedmiller (b13) 2013 Dong, Huang, Zhuang (b3) 2020 Mccue (b12) 2016; 36 Peimin, Huajun, Xinchi (b15) 2018 Schulman, Levine, Moritz, Jordan, Abbeel (b17) 2015 Xu (10.1016/j.oceaneng.2023.115943_b26) 2022 Du (10.1016/j.oceaneng.2023.115943_b4) 2022; 246 10.1016/j.oceaneng.2023.115943_b31 Wang (10.1016/j.oceaneng.2023.115943_b20) 2022; 248 10.1016/j.oceaneng.2023.115943_b18 Xu (10.1016/j.oceaneng.2023.115943_b28) 2020; 73 Cui (10.1016/j.oceaneng.2023.115943_b2) 2022; 18 Mnih (10.1016/j.oceaneng.2023.115943_b13) 2013 Schulman (10.1016/j.oceaneng.2023.115943_b17) 2015 Zhang (10.1016/j.oceaneng.2023.115943_b30) 2018; 151 Gonzalez-Garcia (10.1016/j.oceaneng.2023.115943_b7) 2020 Wang (10.1016/j.oceaneng.2023.115943_b21) 2020; 32 Wang (10.1016/j.oceaneng.2023.115943_b22) 2023; 270 Li (10.1016/j.oceaneng.2023.115943_b9) 2017; 267 Riani (10.1016/j.oceaneng.2023.115943_b16) 2018; 75 Fujimoto (10.1016/j.oceaneng.2023.115943_b6) 2018 Kien (10.1016/j.oceaneng.2023.115943_b8) 2021; 51 Sutton (10.1016/j.oceaneng.2023.115943_b19) 1998 Wang (10.1016/j.oceaneng.2023.115943_b23) 2019 Peimin (10.1016/j.oceaneng.2023.115943_b15) 2018 Woo (10.1016/j.oceaneng.2023.115943_b25) 2019; 183 Wang (10.1016/j.oceaneng.2023.115943_b24) 2016; 3 Lillicrap (10.1016/j.oceaneng.2023.115943_b10) 2015 Liu (10.1016/j.oceaneng.2023.115943_b11) 2022 Zhao (10.1016/j.oceaneng.2023.115943_b33) 2020; 22 Niu (10.1016/j.oceaneng.2023.115943_b14) 2021; 632 Yi (10.1016/j.oceaneng.2023.115943_b29) 2020; 112 Mccue (10.1016/j.oceaneng.2023.115943_b12) 2016; 36 Chen (10.1016/j.oceaneng.2023.115943_b1) 2022 Dong (10.1016/j.oceaneng.2023.115943_b3) 2020 Xu (10.1016/j.oceaneng.2023.115943_b27) 2020; 217 Fu (10.1016/j.oceaneng.2023.115943_b5) 2022 Zhao (10.1016/j.oceaneng.2023.115943_b32) 2021; 32 |
| References_xml | – year: 2022 ident: b5 article-title: Finite-time coordinated path following control of underactuated surface vehicles based on event-triggered mechanism publication-title: Ocean Eng. – volume: 112 year: 2020 ident: b29 article-title: Orbital stabilization of nonlinear systems via mexican sombrero energy shaping and pumping-and-damping injection publication-title: Automatica – year: 2015 ident: b10 article-title: Continuous control with deep reinforcement learning publication-title: Comput. Sci. – start-page: 1 year: 2019 end-page: 5 ident: b23 article-title: Adaptive DDPG design-based sliding-mode control for autonomous underwater vehicles at different speeds publication-title: 2019 IEEE Underwater Technology (UT) – volume: 267 start-page: 466 year: 2017 end-page: 474 ident: b9 article-title: Adaptive NN-DSC control design for path following of underactuated surface vessels with input saturation publication-title: Neurocomputing – year: 1998 ident: b19 article-title: Reinforcement learning:an introduction – start-page: 32 year: 2022 ident: b11 article-title: Robust adaptive self-structuring neural networks tracking control of unmanned surface vessels with uncertainties and time-varying disturbances publication-title: Internat. J. Robust Nonlinear Control – start-page: 1408 year: 2018 end-page: 1413 ident: b15 article-title: The design of gain scheduling PID controller of the USV course control system publication-title: 2018 Chinese Automation Congress (CAC) – volume: 248 year: 2022 ident: b20 article-title: A dynamic collision avoidance solution scheme of unmanned surface vessels based on proactive velocity obstacle and set-based guidance publication-title: Ocean Eng. – volume: 32 start-page: 5468 year: 2021 end-page: 5478 ident: b32 article-title: USV formation and path-following control via deep reinforcement learning with random braking publication-title: IEEE Trans. Neural Netw. Learn. Syst. – volume: 246 year: 2022 ident: b4 article-title: Safe deep reinforcement learning-based adaptive control for USV interception mission publication-title: Ocean Eng. – volume: 632 year: 2021 ident: b14 article-title: Model-free path following control of unmanned surface vehicle based on adaptive line-of-sight guidance publication-title: IOP Conf. Ser. Earth Environ. Sci. – volume: 217 year: 2020 ident: b27 article-title: Intelligent collision avoidance algorithms for USVs via deep reinforcement learning under COLREGs publication-title: Ocean Eng. – year: 2018 ident: b6 article-title: Addressing function approximation error in actor-critic methods – reference: Silver, D., Lever, G., Heess, N., Degris, T., Riedmiller, M., 2014. Deterministic Policy Gradient Algorithms. In: International Conference on Machine Learning. – volume: 51 start-page: 527 year: 2021 end-page: 548 ident: b8 article-title: Adaptive inverse multilayer fuzzy control for uncertain nonlinear system optimizing with differential evolution algorithm publication-title: Appl. Intell. – reference: Zhang, G., Zeng, Q., Zhu, Z., Dai, X., Zhu, C., 2018a. Research on underwater safety inspection and operational robot motion control. In: 2018 33rd Youth Academic Annual Conference of Chinese Association of Automation (YAC). – volume: 32 start-page: 3034 year: 2020 end-page: 3045 ident: b21 article-title: Reinforcement learning-based optimal tracking control of an unknown unmanned surface vehicle publication-title: IEEE Trans. Neural Netw. Learn. Syst. – year: 2013 ident: b13 article-title: Playing atari with deep reinforcement learning publication-title: Comput. Sci. – start-page: 1 year: 2020 end-page: 7 ident: b7 article-title: Usv path-following control based on deep reinforcement learning and adaptive control publication-title: Global Oceans 2020: Singapore–US Gulf Coast – start-page: 3547 year: 2020 end-page: 3551 ident: b3 article-title: Heading angle controller design of USV based on improved sliding mode active disturbance rejection control publication-title: 2020 Chinese Automation Congress (CAC) – volume: 36 start-page: 78 year: 2016 end-page: 79 ident: b12 article-title: Handbook of marine craft hydrodynamics and motion control publication-title: IEEE Control Syst. – volume: 18 start-page: 6950 year: 2022 end-page: 6961 ident: b2 article-title: Filtered probabilistic model predictive control-based reinforcement learning for unmanned surface vehicles publication-title: IEEE Trans. Ind. Inform. – volume: 75 start-page: 108 year: 2018 end-page: 117 ident: b16 article-title: Adaptive integral terminal sliding mode control for upper-limb rehabilitation exoskeleton publication-title: Control Eng. Pract. – volume: 151 start-page: 82 year: 2018 end-page: 92 ident: b30 article-title: A novel adaptive second order sliding mode path following control for a portable AUV publication-title: Ocean Eng. – volume: 22 start-page: 6208 year: 2020 end-page: 6220 ident: b33 article-title: Path following optimization for an underactuated USV using smoothly-convergent deep reinforcement learning publication-title: IEEE Trans. Intell. Transp. Syst. – volume: 73 start-page: 1306 year: 2020 end-page: 1325 ident: b28 article-title: Dynamic collision avoidance algorithm for unmanned surface vehicles via layered artificial potential field with collision cone publication-title: J. Navig. – start-page: 1889 year: 2015 end-page: 1897 ident: b17 article-title: Trust region policy optimization publication-title: Comput. Sci. – volume: 3 start-page: 113 year: 2016 end-page: 120 ident: b24 article-title: Where does alphago go: From church-turing thesis to alphago thesis and beyond publication-title: ACTA Autom. Sin. – year: 2022 ident: b1 article-title: The research on evasion strategy of unpowered aircraft based on deep reinforcement learning – volume: 270 year: 2023 ident: b22 article-title: Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments publication-title: Ocean Eng. – year: 2022 ident: b26 article-title: COLREGs-abiding hybrid collision avoidance algorithm based on deep reinforcement learning for USVs publication-title: Ocean Eng. – volume: 183 start-page: 155 year: 2019 end-page: 166 ident: b25 article-title: Deep reinforcement learning-based controller for path following of an unmanned surface vehicle publication-title: Ocean Eng. – issue: 246-Feb.15 year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b5 article-title: Finite-time coordinated path following control of underactuated surface vehicles based on event-triggered mechanism publication-title: Ocean Eng. – start-page: 1408 year: 2018 ident: 10.1016/j.oceaneng.2023.115943_b15 article-title: The design of gain scheduling PID controller of the USV course control system – volume: 51 start-page: 527 issue: 1 year: 2021 ident: 10.1016/j.oceaneng.2023.115943_b8 article-title: Adaptive inverse multilayer fuzzy control for uncertain nonlinear system optimizing with differential evolution algorithm publication-title: Appl. Intell. doi: 10.1007/s10489-020-01819-9 – volume: 217 issue: 3 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b27 article-title: Intelligent collision avoidance algorithms for USVs via deep reinforcement learning under COLREGs publication-title: Ocean Eng. – ident: 10.1016/j.oceaneng.2023.115943_b18 – start-page: 3547 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b3 article-title: Heading angle controller design of USV based on improved sliding mode active disturbance rejection control – start-page: 32 issue: 6 year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b11 article-title: Robust adaptive self-structuring neural networks tracking control of unmanned surface vessels with uncertainties and time-varying disturbances publication-title: Internat. J. Robust Nonlinear Control – volume: 632 year: 2021 ident: 10.1016/j.oceaneng.2023.115943_b14 article-title: Model-free path following control of unmanned surface vehicle based on adaptive line-of-sight guidance publication-title: IOP Conf. Ser. Earth Environ. Sci. doi: 10.1088/1755-1315/632/3/032046 – volume: 112 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b29 article-title: Orbital stabilization of nonlinear systems via mexican sombrero energy shaping and pumping-and-damping injection publication-title: Automatica doi: 10.1016/j.automatica.2019.108661 – issue: 247-Mar.1 year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b26 article-title: COLREGs-abiding hybrid collision avoidance algorithm based on deep reinforcement learning for USVs publication-title: Ocean Eng. – year: 2015 ident: 10.1016/j.oceaneng.2023.115943_b10 article-title: Continuous control with deep reinforcement learning publication-title: Comput. Sci. – volume: 73 start-page: 1306 issue: 6 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b28 article-title: Dynamic collision avoidance algorithm for unmanned surface vehicles via layered artificial potential field with collision cone publication-title: J. Navig. doi: 10.1017/S0373463320000284 – volume: 151 start-page: 82 issue: MAR.1 year: 2018 ident: 10.1016/j.oceaneng.2023.115943_b30 article-title: A novel adaptive second order sliding mode path following control for a portable AUV publication-title: Ocean Eng. – start-page: 1 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b7 article-title: Usv path-following control based on deep reinforcement learning and adaptive control – volume: 267 start-page: 466 issue: dec.6 year: 2017 ident: 10.1016/j.oceaneng.2023.115943_b9 article-title: Adaptive NN-DSC control design for path following of underactuated surface vessels with input saturation publication-title: Neurocomputing doi: 10.1016/j.neucom.2017.06.042 – volume: 75 start-page: 108 issue: jun. year: 2018 ident: 10.1016/j.oceaneng.2023.115943_b16 article-title: Adaptive integral terminal sliding mode control for upper-limb rehabilitation exoskeleton publication-title: Control Eng. Pract. doi: 10.1016/j.conengprac.2018.02.013 – volume: 22 start-page: 6208 issue: 10 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b33 article-title: Path following optimization for an underactuated USV using smoothly-convergent deep reinforcement learning publication-title: IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2020.2989352 – start-page: 1889 year: 2015 ident: 10.1016/j.oceaneng.2023.115943_b17 article-title: Trust region policy optimization publication-title: Comput. Sci. – volume: 270 year: 2023 ident: 10.1016/j.oceaneng.2023.115943_b22 article-title: Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments publication-title: Ocean Eng. doi: 10.1016/j.oceaneng.2023.113670 – year: 2018 ident: 10.1016/j.oceaneng.2023.115943_b6 – volume: 36 start-page: 78 issue: 1 year: 2016 ident: 10.1016/j.oceaneng.2023.115943_b12 article-title: Handbook of marine craft hydrodynamics and motion control publication-title: IEEE Control Syst. doi: 10.1109/MCS.2015.2495095 – year: 1998 ident: 10.1016/j.oceaneng.2023.115943_b19 – volume: 32 start-page: 3034 issue: 7 year: 2020 ident: 10.1016/j.oceaneng.2023.115943_b21 article-title: Reinforcement learning-based optimal tracking control of an unknown unmanned surface vehicle publication-title: IEEE Trans. Neural Netw. Learn. Syst. doi: 10.1109/TNNLS.2020.3009214 – ident: 10.1016/j.oceaneng.2023.115943_b31 – volume: 18 start-page: 6950 issue: 10 year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b2 article-title: Filtered probabilistic model predictive control-based reinforcement learning for unmanned surface vehicles publication-title: IEEE Trans. Ind. Inform. doi: 10.1109/TII.2022.3142323 – volume: 32 start-page: 5468 issue: 12 year: 2021 ident: 10.1016/j.oceaneng.2023.115943_b32 article-title: USV formation and path-following control via deep reinforcement learning with random braking publication-title: IEEE Trans. Neural Netw. Learn. Syst. doi: 10.1109/TNNLS.2021.3068762 – volume: 248 year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b20 article-title: A dynamic collision avoidance solution scheme of unmanned surface vessels based on proactive velocity obstacle and set-based guidance publication-title: Ocean Eng. – volume: 3 start-page: 113 issue: 2 year: 2016 ident: 10.1016/j.oceaneng.2023.115943_b24 article-title: Where does alphago go: From church-turing thesis to alphago thesis and beyond publication-title: ACTA Autom. Sin. – volume: 246 year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b4 article-title: Safe deep reinforcement learning-based adaptive control for USV interception mission publication-title: Ocean Eng. doi: 10.1016/j.oceaneng.2021.110477 – volume: 183 start-page: 155 year: 2019 ident: 10.1016/j.oceaneng.2023.115943_b25 article-title: Deep reinforcement learning-based controller for path following of an unmanned surface vehicle publication-title: Ocean Eng. doi: 10.1016/j.oceaneng.2019.04.099 – year: 2022 ident: 10.1016/j.oceaneng.2023.115943_b1 – year: 2013 ident: 10.1016/j.oceaneng.2023.115943_b13 article-title: Playing atari with deep reinforcement learning publication-title: Comput. Sci. – start-page: 1 year: 2019 ident: 10.1016/j.oceaneng.2023.115943_b23 article-title: Adaptive DDPG design-based sliding-mode control for autonomous underwater vehicles at different speeds |
| SSID | ssj0006603 |
| Score | 2.4108062 |
| Snippet | This paper addresses cooperative control of velocity and heading for an unmanned surface vessel (USV) utilizing a twin delay deep deterministic policy gradient... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 115943 |
| SubjectTerms | Neural network Reinforcement learning Twin delay deep deterministic policy gradient Unmanned surface vessel |
| Title | Cooperative control of velocity and heading for unmanned surface vessel based on twin delayed deep deterministic policy gradient with an integral compensator |
| URI | https://dx.doi.org/10.1016/j.oceaneng.2023.115943 |
| Volume | 288 |
| WOSCitedRecordID | wos001097846700001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-5258 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0006603 issn: 0029-8018 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9NAEF5FKQdAQlBAlJfmwC1yidePtY9VVQQcCqhFCifL3l3TVK4dpUlo_wg3_isz-0gMrVQQ4rJKVpms7Pm8Mzue-YaxV-jl1qqqRcBjlQYxbpJBqUodoLWVXMYx-cSm2YQ4PMwmk_zjYPDd18KsGtG22cVFPvuvqsY5VDaVzv6Futd_ihP4GZWOI6odxz9S_H7XzbTj8_aJ6OgRUm6QJJebIuUnNnXe5Bgu27OSNtvR-XJel_iYr4hOvBmRfVPmXcI3kzHblJf4XWk9w8Hm0BiSZ9PnQV6Ovs5N9tjCl8t5JorGpK3jabm0xMZrX_iDpLcAekOIuAnu2w3oy7Qqp73ItonqHp0sm-7qbz9NqR7ZzbsoBo-onM_WcdrQmi-v2eQy2VKDnEyo3aC13aEzEeHp2fK9-y2c29aAV8yBjUyc7nZ0QXg9u7Q0Wokkt9xQv1FtHxlCelyPUyslLtC0b3GR5NmQbe29O5i8X9v4NB1HPnmIBHq159evdr3b03Nlju-ze-4MAnsWOw_YQLfb7E6PmXKb3TXKcXTmD9mPHqjAgQq6GjyoAEEFDlSAoAIPKnCgAgsqMKCCrgUCFThQAYEKfgEVWFCBBxUQqHAV8KCCHqgesc9vDo733wausUcgo5AvgiyviGcyTGSUVbweh1VUq1hJFYe5GEsdykqHsRYqlQodLBEqPFyFSVnLrM5TJaPHbNh2rX7CQITUcaCSNddpjN5qJQXKR7yMx1rlKtxhib_vhXSs99R8pSl8euNp4fVVkL4Kq68d9notN7O8LzdK5F6thfNerVdaIBpvkH36D7LP2O3NA_WcDRfzpX7BbsnVYno-f-mA-xMKy8uu |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cooperative+control+of+velocity+and+heading+for+unmanned+surface+vessel+based+on+twin+delayed+deep+deterministic+policy+gradient+with+an+integral+compensator&rft.jtitle=Ocean+engineering&rft.au=Wang%2C+Yibai&rft.au=Zhao%2C+Shulong&rft.au=Wang%2C+Qingling&rft.date=2023-11-15&rft.pub=Elsevier+Ltd&rft.issn=0029-8018&rft.eissn=1873-5258&rft.volume=288&rft_id=info:doi/10.1016%2Fj.oceaneng.2023.115943&rft.externalDocID=S0029801823023272 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0029-8018&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0029-8018&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0029-8018&client=summon |