Programming cobots by voice: a pragmatic, web-based approach
This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces and human-machine interfaces, which can be used by...
Gespeichert in:
| Veröffentlicht in: | International journal of computer integrated manufacturing Jg. 36; H. 1; S. 86 - 109 |
|---|---|
| Hauptverfasser: | , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Taylor & Francis
02.01.2023
|
| Schlagworte: | |
| ISSN: | 0951-192X, 1362-3052 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces and human-machine interfaces, which can be used by people with little or no programming experience. The framework follows a meta-programming approach by enabling users to program cobots by voice in addition to using a mouse, tablet, or keyboard. Upon a voice instruction, the framework automates the manual tasks required to manipulate the vendor-provided interfaces. The main advantages of this approach are simplified, guided programming, which only requires the knowledge of 5-10 voice instructions; increased programming speed compared to the manual approach; and the possibility of sharing programs as videos. The approach is generalized to other kinds of robots and robot programming tools using so-called meta-controllers, which leverage the power of graphical user interface automation tools and techniques. |
|---|---|
| AbstractList | This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces and human-machine interfaces, which can be used by people with little or no programming experience. The framework follows a meta-programming approach by enabling users to program cobots by voice in addition to using a mouse, tablet, or keyboard. Upon a voice instruction, the framework automates the manual tasks required to manipulate the vendor-provided interfaces. The main advantages of this approach are simplified, guided programming, which only requires the knowledge of 5-10 voice instructions; increased programming speed compared to the manual approach; and the possibility of sharing programs as videos. The approach is generalized to other kinds of robots and robot programming tools using so-called meta-controllers, which leverage the power of graphical user interface automation tools and techniques. |
| Author | Ionescu, Tudor B. Schlund, Sebastian |
| Author_xml | – sequence: 1 givenname: Tudor B. surname: Ionescu fullname: Ionescu, Tudor B. email: tudor.ionescu@tuwien.ac.at organization: TU Wien – sequence: 2 givenname: Sebastian orcidid: 0000-0002-8142-0255 surname: Schlund fullname: Schlund, Sebastian organization: TU Wien |
| BookMark | eNqFkMtKAzEYhYNUsFYfQZgHcGr-ZNLOqAuleIOCLhTchT-3GpmZDMlg6dvboXXjQldnc74D5zsmoza0lpAzoFOgJb2glQCo2PuUUcamDIpyLooDMgY-Yzmngo3IeOjkQ-mIHKf0SSlwUdIxuX6JYRWxaXy7ynRQoU-Z2mRfwWt7mWHWRVw12Ht9nq2tyhUmazLsuhhQf5yQQ4d1sqf7nJC3-7vXxWO-fH54Wtwuc82B9rlm3CiFlQIB1RygNJVRpXBgTKGEsAJnfF4gLa1TUAJqN6PcFpXlljmKFZ8QsdvVMaQUrZNd9A3GjQQqBwXyR4EcFMi9gi139YvTvt-eCW0f0df_0jc72rcuxAbXIdZG9ripQ3QRW-2T5H9PfAOsT3cl |
| CitedBy_id | crossref_primary_10_3390_app15020726 crossref_primary_10_1109_ACCESS_2024_3504354 crossref_primary_10_1016_j_mfglet_2025_06_145 |
| Cites_doi | 10.1109/ETFA46521.2020.9211886 10.1016/j.procir.2018.03.224 10.1109/ROMAN.2011.6005285 10.1016/j.procir.2018.01.019 10.3390/s21082589 10.1016/j.rcim.2015.04.002 10.1145/3173574.3173940 10.1177/0278364904049250 10.23919/IConAC.2019.8895254 10.1016/j.ifacol.2018.08.236 10.1109/MIS.2001.1183338 10.1145/3379247.3379261 10.1109/LRA.2018.2798300 10.1109/ROMAN.2002.1045664 10.1016/j.procir.2019.03.017 10.1007/978-3-540-73331-7_14 10.1145/1622176.1622213 10.1016/j.procir.2018.01.033 10.1109/ICRA40945.2020.9197517 10.1109/VLHCC.2004.47 10.1109/SIMS.2018.8355303 10.1016/j.cirp.2019.05.002 10.1109/ICSTW.2013.19 10.3390/app112210903 10.1016/j.rcim.2020.102085 10.1108/01439910510629244 10.1038/d41586-018-05588-x 10.1145/354324.354362 10.1016/j.cirp.2014.03.017 10.1016/j.engappai.2020.103903 10.1109/ICIA.2005.1635148 10.3390/robotics10010003 10.1109/IEEM45057.2020.9309971 10.1109/LRA.2019.2928782 10.1109/BLOCKS.2017.8120406 10.1109/VLHCC.2006.9 10.1109/IACC.2016.83 10.1109/EDOC.2001.950428 10.1007/978-3-030-24781-2_4 10.1109/ICRA.2018.8462882 10.1109/ICRA.2019.8793988 10.3390/s20216358 10.1108/IR-01-2018-0017 10.1145/3466819 10.1109/BLOCKS.2015.7369000 10.1016/B978-0-12-821343-8.00010-1 10.1016/j.procir.2017.03.126 10.1109/RO-MAN47096.2020.9223573 10.1016/j.rcim.2012.11.002 10.1109/IROS40897.2019.8968041 10.1007/s35144-020-0550-2 10.2139/ssrn.3869551 |
| ContentType | Journal Article |
| Copyright | 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. 2022 |
| Copyright_xml | – notice: 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. 2022 |
| DBID | 0YH AAYXX CITATION |
| DOI | 10.1080/0951192X.2022.2148754 |
| DatabaseName | Taylor & Francis Open Access CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: 0YH name: Taylor & Francis Open Access url: https://www.tandfonline.com sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1362-3052 |
| EndPage | 109 |
| ExternalDocumentID | 10_1080_0951192X_2022_2148754 2148754 |
| Genre | Research Article |
| GroupedDBID | .4S .7F .DC .QJ 0BK 0R~ 0YH 29J 30N 4.4 5GY 5VS 8VB AAENE AAGDL AAHIA AAJMT AALDU AAMIU AAPUL AAQRR ABCCY ABFIM ABHAV ABJNI ABLIJ ABPAQ ABPEM ABPPZ ABTAI ABXUL ABXYU ACGEJ ACGFS ACTIO ADCVX ADGTB ADMLS ADXPE AEISY AEMOZ AENEX AEOZL AEPSL AEYOC AFKVX AFRVT AGDLA AGMYJ AHDZW AHQJS AIJEM AIYEW AJWEG AKBVH AKOOK AKVCP ALMA_UNASSIGNED_HOLDINGS ALQZU AQRUH AQTUD ARCSS AVBZW AWYRJ BLEHA CCCUG CE4 CS3 DGEBU DKSSO EBO EBR EBS EBU ECS EDO EMK EPL E~A E~B GTTXZ H13 HF~ HZ~ H~P I-F IPNFZ J.P K1G KYCEM LJTGL M4Z MK~ ML~ NA5 NX~ O9- P2P QWB RIG RNANH RNS ROSJB RTWRZ S-T SNACF TASJS TBQAZ TDBHL TEN TFL TFT TFW TH9 TNC TTHFI TUROJ TUS TWF UPT UT5 UU3 ZGOLN ZL0 ~S~ AAYXX CITATION |
| ID | FETCH-LOGICAL-c310t-c23dbba9b15197118d9db85f1dd4b55e5a6374a08efb181acf603e49e3e2f0a93 |
| IEDL.DBID | 0YH |
| ISICitedReferencesCount | 4 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000890391900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0951-192X |
| IngestDate | Tue Nov 18 22:08:44 EST 2025 Sat Nov 29 06:18:29 EST 2025 Mon Oct 20 23:47:54 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | open-access: http://creativecommons.org/licenses/by-nc-nd/4.0/: http://creativecommons.org/licenses/by-nc-nd/4.0/: This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c310t-c23dbba9b15197118d9db85f1dd4b55e5a6374a08efb181acf603e49e3e2f0a93 |
| ORCID | 0000-0002-8142-0255 |
| OpenAccessLink | https://www.tandfonline.com/doi/abs/10.1080/0951192X.2022.2148754 |
| PageCount | 24 |
| ParticipantIDs | informaworld_taylorfrancis_310_1080_0951192X_2022_2148754 crossref_primary_10_1080_0951192X_2022_2148754 crossref_citationtrail_10_1080_0951192X_2022_2148754 |
| PublicationCentury | 2000 |
| PublicationDate | 01/02/2023 |
| PublicationDateYYYYMMDD | 2023-01-02 |
| PublicationDate_xml | – month: 01 year: 2023 text: 01/02/2023 day: 02 |
| PublicationDecade | 2020 |
| PublicationTitle | International journal of computer integrated manufacturing |
| PublicationYear | 2023 |
| Publisher | Taylor & Francis |
| Publisher_xml | – name: Taylor & Francis |
| References | cit0033 Schäfer P. M. (cit0047) 2021; 18 cit0034 cit0031 cit0030 Mohs C. (cit0035) 2006; 11 cit0039 cit0037 cit0038 cit0036 cit0022 cit0023 cit0020 cit0021 cit0028 cit0029 cit0026 cit0027 cit0024 cit0025 cit0011 cit0055 cit0012 cit0056 cit0053 cit0010 cit0054 cit0051 cit0052 cit0050 Lv X. (cit0032) 2008 Wilhelm B. (cit0057) 2017; 594 cit0019 cit0017 cit0018 cit0015 cit0016 cit0013 cit0014 cit0058 cit0044 cit0001 cit0045 cit0042 cit0043 cit0040 cit0041 cit0008 cit0009 cit0006 cit0007 cit0004 cit0048 cit0005 cit0049 cit0002 cit0046 cit0003 |
| References_xml | – ident: cit0020 doi: 10.1109/ETFA46521.2020.9211886 – ident: cit0030 doi: 10.1016/j.procir.2018.03.224 – volume: 18 start-page: 702 issue: 1 year: 2021 ident: cit0047 publication-title: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning – ident: cit0046 doi: 10.1109/ROMAN.2011.6005285 – ident: cit0012 doi: 10.1016/j.procir.2018.01.019 – ident: cit0019 doi: 10.3390/s21082589 – ident: cit0039 doi: 10.1016/j.rcim.2015.04.002 – ident: cit0054 doi: 10.1145/3173574.3173940 – volume: 594 start-page: 3 volume-title: Advances in Human Factors, Business Management and Leadership. AHFE 2017. Advances in Intelligent Systems and Computing year: 2017 ident: cit0057 – ident: cit0049 doi: 10.1177/0278364904049250 – ident: cit0031 doi: 10.23919/IConAC.2019.8895254 – ident: cit0052 doi: 10.1016/j.ifacol.2018.08.236 – ident: cit0040 doi: 10.1109/MIS.2001.1183338 – ident: cit0034 doi: 10.1145/3379247.3379261 – ident: cit0051 doi: 10.1109/LRA.2018.2798300 – ident: cit0044 doi: 10.1109/ROMAN.2002.1045664 – ident: cit0021 doi: 10.1016/j.procir.2019.03.017 – ident: cit0036 doi: 10.1007/978-3-540-73331-7_14 – ident: cit0058 doi: 10.1145/1622176.1622213 – ident: cit0015 doi: 10.1016/j.procir.2018.01.033 – ident: cit0026 doi: 10.1109/ICRA40945.2020.9197517 – ident: cit0024 doi: 10.1109/VLHCC.2004.47 – ident: cit0042 doi: 10.1109/SIMS.2018.8355303 – ident: cit0053 doi: 10.1016/j.cirp.2019.05.002 – volume: 11 start-page: 75 year: 2006 ident: cit0035 publication-title: MMI-Interaktiv – ident: cit0028 doi: 10.1109/ICSTW.2013.19 – ident: cit0041 doi: 10.3390/app112210903 – ident: cit0038 doi: 10.1016/j.rcim.2020.102085 – ident: cit0006 doi: 10.1108/01439910510629244 – ident: cit0037 doi: 10.1038/d41586-018-05588-x – ident: cit0002 doi: 10.1145/354324.354362 – ident: cit0033 doi: 10.1016/j.cirp.2014.03.017 – ident: cit0005 doi: 10.1016/j.engappai.2020.103903 – ident: cit0029 doi: 10.1109/ICIA.2005.1635148 – ident: cit0017 doi: 10.3390/robotics10010003 – ident: cit0048 doi: 10.1109/IEEM45057.2020.9309971 – ident: cit0050 doi: 10.1109/LRA.2019.2928782 – ident: cit0055 doi: 10.1109/BLOCKS.2017.8120406 – ident: cit0003 doi: 10.1109/VLHCC.2006.9 – ident: cit0025 doi: 10.1109/IACC.2016.83 – start-page: 2490 volume-title: 2008 IEEE International Conference on Automation and Logistics year: 2008 ident: cit0032 – ident: cit0007 doi: 10.3390/app112210903 – ident: cit0027 doi: 10.1109/EDOC.2001.950428 – ident: cit0004 doi: 10.1007/978-3-030-24781-2_4 – ident: cit0014 doi: 10.1109/ICRA.2018.8462882 – ident: cit0016 – ident: cit0011 doi: 10.1109/ICRA.2019.8793988 – ident: cit0022 doi: 10.3390/s20216358 – ident: cit0043 doi: 10.1108/IR-01-2018-0017 – ident: cit0001 doi: 10.1145/3466819 – ident: cit0010 doi: 10.1109/BLOCKS.2015.7369000 – ident: cit0009 doi: 10.1016/B978-0-12-821343-8.00010-1 – ident: cit0013 doi: 10.1016/j.procir.2017.03.126 – ident: cit0018 doi: 10.1109/RO-MAN47096.2020.9223573 – ident: cit0045 doi: 10.1016/j.rcim.2012.11.002 – ident: cit0056 doi: 10.1109/IROS40897.2019.8968041 – ident: cit0008 doi: 10.1007/s35144-020-0550-2 – ident: cit0023 doi: 10.2139/ssrn.3869551 |
| SSID | ssj0013580 |
| Score | 2.3243566 |
| Snippet | This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is... |
| SourceID | crossref informaworld |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 86 |
| SubjectTerms | cobots GUI automation meta-programming plugins speech recognition Voice-based programming |
| Title | Programming cobots by voice: a pragmatic, web-based approach |
| URI | https://www.tandfonline.com/doi/abs/10.1080/0951192X.2022.2148754 |
| Volume | 36 |
| WOSCitedRecordID | wos000890391900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAWR databaseName: Taylor & Francis Online Journals customDbUrl: eissn: 1362-3052 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0013580 issn: 0951-192X databaseCode: TFW dateStart: 19880101 isFulltext: true titleUrlDefault: https://www.tandfonline.com providerName: Taylor & Francis |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV09T8MwED1BYYCBb0T5kgdGUhLHSW3EghBVp6pDEWWK_ImQoKmaUIl_j-0kUAZggCVShouii31-zzm_B3BGNVNRKnlgJOMBYZwFwtZiZ2YmCaOE01B6s4nuYEDHYzasuwmLuq3ScWhTCUX4Wu0mNxdF0xF34VCBBSZjy-4w7uDIYW6yDCvYUhPHv8KH_uePhMSbp3k3eRfTHOL57jFflqcv4qULy05v8x9eeAs2asyJrqtBsg1LerID6wtKhLtwNaz6tF7sHZK5yMsCiTc0z20duUQcTWf80Yu7niNbdwO39inU6JHvwV3vdnTTD2pjhUBaNFcGEsdKCM5E5I6tWoqhmBI0MZFSRCSJTngadwkPqTbCIgAuTRrGmjAda2xCzuJ9aE3yiT4AJIXilGAsIsOIJpxbgJAKRnWoYsv1cBtIk89M1qrjzvziOYsacdI6OZlLTlYnpw2dj7BpJbvxWwBb_FhZ6fc7TGVOksU_xh7-IfYI1pwBvd-UwcfQKmev-gRW5bx8Kmanfiza66h3_w4jbNru |
| linkProvider | Taylor & Francis |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwED5BQQIG3ojy9MBIIHGcYCMWhKiKKBVDEd0iv4KQoKnaUIl_j-0kkA7AAGOGi6KLc_fd5e77AI6oZiqIJfdSybhHGGeeMLHYiplJwijh1JdObOKs26X9PqvvwtixSltDpwVRhIvV9uO2zehqJO7UwgKDTPqmvMP4BAcWdJNZmItMrrX8-b3W49efhMippzk5eWtTbfF8d5up_DTFXlrLO62V_3jiVVguUSe6LI7JGszowTos1bgIN-DivpjUejVXSGYiy8dIvKNJZiLJOeJoOOJPjt71GJnI69nsp1DFSL4JD63r3lXbK6UVPGnwXO5JHCohOBOBXVw1RYZiStAoDZQiIop0xGPjSu5TnQqDAbhMYz_UhOlQ49TnLNyCxiAb6G1AUihOCcYiSBnRhHMDEWLBqPZVaKo93ARSOTSRJe-4lb94SYKKnrR0TmKdk5TOacLJp9mwIN74zYDV31aSu45HWsiTJOGPtjt_sD2EhXbvrpN0brq3u7Bo5ehdiwbvQSMfvel9mJeT_Hk8OnAH8wN4JN4G |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwED5BQQgG3ojy9MBISuI4aYxYEBCBQFWHIrpFfiIkaKo2VOLfYzsJlAEYYMxwUXS-3H1nn78P4ChRVAaxYJ4WlHmEMupxk4utmJkgNCEs8YUTm2h3Okm_T7vVNOG4Gqu0PbQuiSJcrrY_91DqeiLuxKICA0z6prvDuIUDi7nJLMwZ6BzbIO-lD58HCZETT3Nq8tamvsTz3Wu-lKcv5KVTZSdd-YcPXoXlCnOi8zJI1mBGDdZhaYqJcAPOuuWc1ot5QiLneTFG_A1NcpNHThFDwxF7dOSux8jkXc_WPolqPvJNuE-vehfXXiWs4AmD5gpP4FByzigP7LVV02JIKnkS6UBKwqNIRSwO24T5idLcIAAmdOyHilAVKqx9RsMtaAzygdoGJLhkCcGYB5oSRRgzACHmNFG-DE2vh5tAan9momIdt-IXz1lQk5NWzsmsc7LKOU1ofZgNS9qN3wzo9GJlhdvv0KU4SRb-aLvzB9tDWOheptndTed2FxatFr3bn8F70ChGr2of5sWkeBqPDlxYvgPrhdy4 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Programming+cobots+by+voice%3A+a+pragmatic%2C+web-based+approach&rft.jtitle=International+journal+of+computer+integrated+manufacturing&rft.au=Ionescu%2C+Tudor+B.&rft.au=Schlund%2C+Sebastian&rft.date=2023-01-02&rft.pub=Taylor+%26+Francis&rft.issn=0951-192X&rft.eissn=1362-3052&rft.volume=36&rft.issue=1&rft.spage=86&rft.epage=109&rft_id=info:doi/10.1080%2F0951192X.2022.2148754&rft.externalDBID=0YH&rft.externalDocID=2148754 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0951-192X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0951-192X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0951-192X&client=summon |