Evaluating Task Optimization and Reinforcement Learning Models in Robotic Task Parameterization

The rapid evolution of industrial robot hardware has created a technological gap with software, limiting its adoption. The software solutions proposed in recent years have yet to meet the industrial sector's requirements, as they focus more on the definition of task structure than the definitio...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE access Vol. 12; pp. 173734 - 173748
Main Authors:	Delledonne, Michele, Villagrossi, Enrico, Beschi, Manuel, Rastegarpanah, Alireza
Format:	Journal Article
Language:	English
Published:	Piscataway IEEE 2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Artificial intelligence Efficiency Flexibility Industrial robots intuitive robot programming Libraries Machine learning Mathematical models Modular structures Optimization Parameterization Parameters Programming Reinforcement learning Robot learning Robot sensing systems robotic task optimization Robots Service robots Software task-oriented programming
ISSN:	2169-3536, 2169-3536
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	The rapid evolution of industrial robot hardware has created a technological gap with software, limiting its adoption. The software solutions proposed in recent years have yet to meet the industrial sector's requirements, as they focus more on the definition of task structure than the definition and tuning of its execution parameters. A framework for task parameter optimization was developed to address this gap. It breaks down the task using a modular structure, allowing the task optimization piece by piece. The optimization is performed with a dedicated hill-climbing algorithm. This paper revisits the framework by proposing an alternative approach that replaces the algorithmic component with reinforcement learning (RL) models. Five RL models are proposed with increasing complexity and efficiency. A comparative analysis of the traditional algorithm and RL models is presented, highlighting efficiency, flexibility, and usability. The results demonstrate that although RL models improve task optimization efficiency by 95%, they still need more flexibility. However, the nature of these models provides significant opportunities for future advancements.
AbstractList	The rapid evolution of industrial robot hardware has created a technological gap with software, limiting its adoption. The software solutions proposed in recent years have yet to meet the industrial sector’s requirements, as they focus more on the definition of task structure than the definition and tuning of its execution parameters. A framework for task parameter optimization was developed to address this gap. It breaks down the task using a modular structure, allowing the task optimization piece by piece. The optimization is performed with a dedicated hill-climbing algorithm. This paper revisits the framework by proposing an alternative approach that replaces the algorithmic component with reinforcement learning (RL) models. Five RL models are proposed with increasing complexity and efficiency. A comparative analysis of the traditional algorithm and RL models is presented, highlighting efficiency, flexibility, and usability. The results demonstrate that although RL models improve task optimization efficiency by 95%, they still need more flexibility. However, the nature of these models provides significant opportunities for future advancements.
Author	Delledonne, Michele Beschi, Manuel Rastegarpanah, Alireza Villagrossi, Enrico
Author_xml	– sequence: 1 givenname: Michele orcidid: 0000-0001-5236-2706 surname: Delledonne fullname: Delledonne, Michele email: michele.delledonne@stiima.cnr.it organization: Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Milan, Italy – sequence: 2 givenname: Enrico orcidid: 0000-0002-9493-4175 surname: Villagrossi fullname: Villagrossi, Enrico organization: Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Milan, Italy – sequence: 3 givenname: Manuel orcidid: 0000-0002-8845-2313 surname: Beschi fullname: Beschi, Manuel organization: Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Milan, Italy – sequence: 4 givenname: Alireza orcidid: 0000-0003-4264-6857 surname: Rastegarpanah fullname: Rastegarpanah, Alireza organization: School of Metallurgy and Materials, University of Birmingham, Birmingham, U.K
BookMark	eNpNkU9LAzEQxYMoWKufQA8LnluTJtndHEupWqhU_HMOs9lJSW2Tmt0K-ulN3SKdywyP-b0ZeBfk1AePhFwzOmSMqrvxZDJ9fR2O6EgMuaSCS3FCeiOWqwGXPD89ms_JVdOsaKoySbLoET39gvUOWueX2Rs0H9li27qN-0lK8Bn4OntB522IBjfo22yOEP1--SnUuG4y57OXUIXWmQ5_hggbbDEeLC7JmYV1g1eH3ifv99O3yeNgvniYTcbzgeFStYNiRKlhIG2lqBUIqjRQSSYLzGukeZVTI0ssSrCFkCW3xtqSCURl8spKW_M-mXW-dYCV3ka3gfitAzj9J4S41BDTl2vUKGWVq5LRshaCGgBQkglpCs5rVWOVvG47r20MnztsWr0Ku-jT-5ozzkViU-sT3m2ZGJomov2_yqjeB6O7YPQ-GH0IJlE3HeUQ8YgopMoF5b_kr4zS
CODEN	IAECCG
Cites_doi	10.1080/0951192X.2022.2148754 10.1007/s10845-023-02211-3 10.1007/s13218-019-00595-0 10.1145/3466819 10.4236/ahs.2019.81002 10.1515/zwf-2021-0044 10.1007/s35724-022-1138-6 10.3390/app12063164 10.1145/3640008 10.1007/s10723-022-09618-x 10.3390/s23073762 10.1109/TITS.2020.3046478 10.14569/ijacsa.2021.0121070 10.3390/robotics10010050 10.1108/ir-02-2021-0043 10.1016/j.mechatronics.2018.02.009 10.1109/ACCESS.2020.3027152 10.1109/SII58957.2024.10417267 10.1109/ETFA54631.2023.10275720 10.1016/j.compbiomed.2022.106060 10.1016/j.cogr.2023.04.001 10.1007/s10845-023-02294-y 10.1109/TRO.2023.3258669 10.3390/app12020937 10.1016/j.eswa.2023.120254 10.3390/app13042582 10.1007/s10845-023-02096-2 10.1016/j.neucom.2023.126896
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA
DOI	10.1109/ACCESS.2024.3504354
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE/IET Electronic Library CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Directory of Open Access Journals
DatabaseTitle	CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional
DatabaseTitleList	Materials Research Database
Database_xml	– sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2169-3536
EndPage	173748
ExternalDocumentID	oai_doaj_org_article_e55b698108d440caaa95145c733d9deb 10_1109_ACCESS_2024_3504354 10759640
Genre	orig-research
GrantInformation_xml	– fundername: REBELION Project grantid: 101104241 – fundername: Lombardy, Italy Regional Project EcoCirc (deliberation XI/4730 of the 17/05/2021)
GroupedDBID	0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c359t-7200c1a5fb90f4ea98cab5157e6de06b60c58e78af74583fcff814ee9c6bf5fd3
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001409526500009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2169-3536
IngestDate	Fri Oct 03 12:52:06 EDT 2025 Mon Jun 30 12:58:28 EDT 2025 Sat Nov 29 04:27:13 EST 2025 Wed Aug 27 03:03:19 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://creativecommons.org/licenses/by/4.0/legalcode
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c359t-7200c1a5fb90f4ea98cab5157e6de06b60c58e78af74583fcff814ee9c6bf5fd3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-4264-6857 0000-0002-9493-4175 0000-0002-8845-2313 0000-0001-5236-2706
OpenAccessLink	https://ieeexplore.ieee.org/document/10759640
PQID	3133498113
PQPubID	4845423
PageCount	15
ParticipantIDs	ieee_primary_10759640 crossref_primary_10_1109_ACCESS_2024_3504354 doaj_primary_oai_doaj_org_article_e55b698108d440caaa95145c733d9deb proquest_journals_3133498113
PublicationCentury	2000
PublicationDate	20240000 2024-00-00 20240101 2024-01-01
PublicationDateYYYYMMDD	2024-01-01
PublicationDate_xml	– year: 2024 text: 20240000
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE access
PublicationTitleAbbrev	Access
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	Bilancia (ref6) 2023; 13 ref13 ref12 ref15 ref14 ref30 ref11 ref10 Ionescu (ref7) 2021; 10 ref2 ref1 Blankemeyer (ref16); 76 Raffaeli (ref3) 2022; 12 Arents (ref19) 2022; 12 Towers (ref33) 2023 Truong (ref27) 2023; 562 Tsarouchi (ref17); 23 Islam (ref23) 2022; 149 ref24 Han (ref31) 2023; 23 Villani (ref18) 2018; 55 ref25 Coumans (ref32) 2016 ref22 ref28 ref29 ref8 ref9 ref4 Soori (ref20) 2023; 3 ref5 Liu (ref21) 2023; 227 Raffin (ref34) 2021; 22 Chen (ref26) 2021; 10
References_xml	– volume: 23 start-page: 47 volume-title: Proc. CIRP Conf. Assem. Technol. Syst. ident: ref17 article-title: Robotized assembly process using dual arm robot – ident: ref12 doi: 10.1080/0951192X.2022.2148754 – ident: ref24 doi: 10.1007/s10845-023-02211-3 – volume: 10 start-page: 1 issue: 1 year: 2021 ident: ref7 article-title: Leveraging graphical user interface automation for generic robot programming publication-title: Robot. – ident: ref5 doi: 10.1007/s13218-019-00595-0 – ident: ref15 doi: 10.1145/3466819 – ident: ref1 doi: 10.4236/ahs.2019.81002 – ident: ref8 doi: 10.1515/zwf-2021-0044 – volume: 76 start-page: 155 volume-title: Proc. CIRP ident: ref16 article-title: Intuitive robot programming using augmented reality – ident: ref2 doi: 10.1007/s35724-022-1138-6 – volume-title: Pybullet, a Python Module for Physics Simulation for Games, Robotics and Machine Learning year: 2016 ident: ref32 – volume: 12 start-page: 3164 issue: 6 year: 2022 ident: ref3 article-title: Engineering method and tool for the complete virtual commissioning of robotic cells publication-title: Appl. Sci. doi: 10.3390/app12063164 – ident: ref14 doi: 10.1145/3640008 – ident: ref11 doi: 10.1007/s10723-022-09618-x – volume: 23 start-page: 3762 issue: 7 year: 2023 ident: ref31 article-title: A survey on deep reinforcement learning algorithms for robotic manipulation publication-title: Sensors doi: 10.3390/s23073762 – ident: ref22 doi: 10.1109/TITS.2020.3046478 – ident: ref25 doi: 10.14569/ijacsa.2021.0121070 – volume: 10 start-page: 50 issue: 1 year: 2021 ident: ref26 article-title: Industrial robot trajectory tracking control using multi-layer neural networks trained by iterative learning control publication-title: Robotics doi: 10.3390/robotics10010050 – volume-title: Gymnasium year: 2023 ident: ref33 – ident: ref9 doi: 10.1108/ir-02-2021-0043 – volume: 55 start-page: 248 year: 2018 ident: ref18 article-title: Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications publication-title: Mechatronics doi: 10.1016/j.mechatronics.2018.02.009 – ident: ref30 doi: 10.1109/ACCESS.2020.3027152 – ident: ref29 doi: 10.1109/SII58957.2024.10417267 – ident: ref4 doi: 10.1109/ETFA54631.2023.10275720 – volume: 149 year: 2022 ident: ref23 article-title: Human activity recognition using tools of convolutional neural networks: A state of the art review, data sets, challenges, and future prospects publication-title: Comput. Biol. Med. doi: 10.1016/j.compbiomed.2022.106060 – volume: 3 start-page: 54 year: 2023 ident: ref20 article-title: Artificial intelligence, machine learning and deep learning in advanced robotics, a review publication-title: Cognit. Robot. doi: 10.1016/j.cogr.2023.04.001 – ident: ref28 doi: 10.1007/s10845-023-02294-y – ident: ref10 doi: 10.1109/TRO.2023.3258669 – volume: 12 start-page: 937 issue: 2 year: 2022 ident: ref19 article-title: Smart industrial robot control trends, challenges and opportunities within manufacturing publication-title: Appl. Sci. doi: 10.3390/app12020937 – volume: 227 year: 2023 ident: ref21 article-title: Path planning techniques for mobile robots: Review and prospect publication-title: Expert Syst. Appl. doi: 10.1016/j.eswa.2023.120254 – volume: 13 start-page: 2582 issue: 4 year: 2023 ident: ref6 article-title: An overview of industrial robots control and programming approaches publication-title: Appl. Sci. doi: 10.3390/app13042582 – ident: ref13 doi: 10.1007/s10845-023-02096-2 – volume: 562 year: 2023 ident: ref27 article-title: Neural network-based sliding mode controllers applied to robot manipulators: A review publication-title: Neurocomputing doi: 10.1016/j.neucom.2023.126896 – volume: 22 start-page: 1 issue: 268 year: 2021 ident: ref34 article-title: Stable-baselines3: Reliable reinforcement learning implementations publication-title: J. Mach. Learn. Res.
SSID	ssj0000816957
Score	2.2999768
Snippet	The rapid evolution of industrial robot hardware has created a technological gap with software, limiting its adoption. The software solutions proposed in...
SourceID	doaj proquest crossref ieee
SourceType	Open Website Aggregation Database Index Database Publisher
StartPage	173734
SubjectTerms	Algorithms Artificial intelligence Efficiency Flexibility Industrial robots intuitive robot programming Libraries Machine learning Mathematical models Modular structures Optimization Parameterization Parameters Programming Reinforcement learning Robot learning Robot sensing systems robotic task optimization Robots Service robots Software task-oriented programming
SummonAdditionalLinks	– databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LTxsxELYQ6gEOVVtAhNLKB45sY-P3EaJEHBBFEVS5WbN-RFFhg0jK78f2OlWqHnrpdbU7s57xvPz4BqEzLyDfr1SNF440PCRTBA-uIcoEWnY6S9O-Hzfq9lbPZuZuq9VXPhPWwwP3ghsGIVppNCXac04cAKScgAunGPPGhzZ730R4q5gqPlhTaYSqMEOUmOHlaJRGlArCC_6NZdguwf8IRQWxv7ZY-csvl2Az-YDe1ywRX_Z_9xHthO4T2t_CDjxAdlxxurs5vofVT_w9Gf9TvVWJofN4GgoqqisLgLgCqc5x7n72uMKLDk-X7TIx6D-_g3xMKyM39yQO0cNkfD-6bmq3hMYxYdaNSvPdURCxNSTyAEY7aFO2ooL0gchWEid0UBqiynul0cWoKQ_BONlGET07QrvdsgvHCDPKacuMM156HiWA5iAVpNieisWUggzQ-UZw9rkHxbClmCDG9nK2Wc62ynmArrJwf7-aEa3Lg6RnW_Vs_6XnATrMqtnip4SRnAzQ6UZXtprfyrJUefNEjLKT_8H7M9rL4-lXXk7R7vrlV_iC3rnX9WL18rXMvDcAkdt- priority: 102 providerName: Directory of Open Access Journals
Title	Evaluating Task Optimization and Reinforcement Learning Models in Robotic Task Parameterization
URI	https://ieeexplore.ieee.org/document/10759640 https://www.proquest.com/docview/3133498113 https://doaj.org/article/e55b698108d440caaa95145c733d9deb
Volume	12
WOSCitedRecordID	wos001409526500009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9wwELZa1AM99EGp2HaLfOBIIMGv-LisFvXQUoSg4mZN7DFCQLbaXXrsb-_YMQiEeugliqL4EX8Ze2bs-YaxnaAgxVeaKihfVxJJFCGAr2pjsck7nTlp389v5vi4vbiwJyVYPcfCIGI-fIZ76Tbv5Ye5v0uuMpJwo6yWZKG_NEYPwVoPDpWUQcIqU5iFmtruT6ZT-giyAQ_knkhMXUo-WX0ySX_JqvJsKs7ry9Hb_-zZO_amKJJ8MiD_nr3AfoO9fkQv-IG5WaHy7i_5GSyv-Q-aH25L4CWHPvBTzMSpPvsIeeFaveQpQdrNkl_1_HTezamBofgJpJNcidx5qGKTnR_NzqZfq5JQofJC2VVlSCR8Ayp2to4SwbYeOlJoDOqAte507VWLpoVo0nZq9DG2jUS0XndRxSA-srV-3uMW46KRTSest0EHGTVAK0EboOWfhoK0lBHbvR9o92vgzXDZ3qitG3BxCRdXcBmxwwTGw6uJ9Do_oFF2RYYcKtVpSw20QcraAwCph1J5I0SwAbsR20zIPGpvAGXExvfYuiKhSyfIOJdUWSM-_aPYZ7aeujj4W8ZsbbW4wy_slf-9uloutrPxTtfvf2bb-Uf8CzZu2zE
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwEB6hggQceLZioYAPHElx6ld8LKtWRSxLVS2oN8uxx1UFZFF329_fseNWRYgDtyiKn1_Gnhl7vgF4F5XP8ZWmiSrwRiKJoo8-NNxYbMtJZ0na931m5vPu5MQe1WD1EguDiOXyGe7kx3KWH5fhIrvKSMKNslqShX5XSbnLx3CtG5dKziFhlancQi23H_amUxoGWYG7ckdkri4l_9h_Ck1_zavy12JcdpiDx__ZtyfwqKqSbG_E_incweEZPLxFMPgc3H4l8x5O2cKvfrCvtEL8qqGXzA-RHWOhTg3FS8gq2-opyynSfq7Y2cCOl_2SGhiLH_l8lyvTO49VbMK3g_3F9LCpKRWaIJRdN4aEIrRepd7yJNHbLvieVBqDOiLXveZBdWg6n0w-UE0hpa6ViDboPqkUxRZsDMsBXwATrWx7YYONOsqkve-k18aTAkBTQXrKBN5fT7T7PTJnuGJxcOtGXFzGxVVcJvAxg3Hzaaa9Li9oll2VIodK9dpSA12UkgfvPSmIUgUjRLQR-wlsZmRutTeCMoHta2xdldGVE2SeS6qsFS__Uewt3D9cfJm52af551fwIHd39L5sw8b6_AJfw71wuT5bnb8pP-IVa_ncUg
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluating+Task+Optimization+and+Reinforcement+Learning+Models+in+Robotic+Task+Parameterization&rft.jtitle=IEEE+access&rft.au=Delledonne%2C+Michele&rft.au=Villagrossi%2C+Enrico&rft.au=Beschi%2C+Manuel&rft.au=Rastegarpanah%2C+Alireza&rft.date=2024&rft.pub=IEEE&rft.eissn=2169-3536&rft.volume=12&rft.spage=173734&rft.epage=173748&rft_id=info:doi/10.1109%2FACCESS.2024.3504354&rft.externalDocID=10759640
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon