Cyber-physical-system for representing a robot end effector
Programming by Demonstration (PbD) is a method to program robots through the performance of a task by humans. Most implementations are online methods that use visual or force feedback of the demonstrator. However, we developed an offline programming approach for PbD with a special input device withi...
Saved in:
| Published in: | Procedia CIRP Vol. 100; pp. 307 - 312 |
|---|---|
| Main Authors: | , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Elsevier B.V
2021
|
| Subjects: | |
| ISSN: | 2212-8271, 2212-8271 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Programming by Demonstration (PbD) is a method to program robots through the performance of a task by humans. Most implementations are online methods that use visual or force feedback of the demonstrator. However, we developed an offline programming approach for PbD with a special input device within an Augmented Reality Environment. Therefore, this paper aims to answer how the characteristics and functionality of the end effector of a jointed-arm robot can be represented by a haptic input device in order to perform PbD. The PbD process is first carried out on a digital twin of the robot, visualized to the user in real physical space by means of augmented reality technology. The programming of the digital twin can later be transferred to the real robot. The haptic input device in this context is the main part of the Cyber-Physical-System (CPS), which enables the user to interact with the virtual robot. Therefore, the specification of the mechanical and software components of the CPS is of main importance. Within this paper, strategies for the implementation of shape and function abstraction, as well as for ensuring communication, have been worked out. The physical shape of the CPS is kept generic and is only subject to ergonomic restrictions. However, Augmented Reality overlays the physical shape with an exact digital image of the end effector used later in the process. Nevertheless, the physical characteristics of the real robot should be represented as real as possible by the CPS. Therefore, the CPS is equipped with various sensors and actuators. With the CPS it is possible to determine contact forces and to manipulate objects to a certain extent in order to teach gripping strategies to the digital twin. An operating system was developed for communication and control of the electronic components. For the validation of the functionality of the CPS an exemplary PbD process was developed, the results were analyzed and evaluated. |
|---|---|
| AbstractList | Programming by Demonstration (PbD) is a method to program robots through the performance of a task by humans. Most implementations are online methods that use visual or force feedback of the demonstrator. However, we developed an offline programming approach for PbD with a special input device within an Augmented Reality Environment. Therefore, this paper aims to answer how the characteristics and functionality of the end effector of a jointed-arm robot can be represented by a haptic input device in order to perform PbD. The PbD process is first carried out on a digital twin of the robot, visualized to the user in real physical space by means of augmented reality technology. The programming of the digital twin can later be transferred to the real robot. The haptic input device in this context is the main part of the Cyber-Physical-System (CPS), which enables the user to interact with the virtual robot. Therefore, the specification of the mechanical and software components of the CPS is of main importance. Within this paper, strategies for the implementation of shape and function abstraction, as well as for ensuring communication, have been worked out. The physical shape of the CPS is kept generic and is only subject to ergonomic restrictions. However, Augmented Reality overlays the physical shape with an exact digital image of the end effector used later in the process. Nevertheless, the physical characteristics of the real robot should be represented as real as possible by the CPS. Therefore, the CPS is equipped with various sensors and actuators. With the CPS it is possible to determine contact forces and to manipulate objects to a certain extent in order to teach gripping strategies to the digital twin. An operating system was developed for communication and control of the electronic components. For the validation of the functionality of the CPS an exemplary PbD process was developed, the results were analyzed and evaluated. |
| Author | Müller, Fabian Koch, Michael Deuerlein, Christian |
| Author_xml | – sequence: 1 givenname: Fabian surname: Müller fullname: Müller, Fabian email: fabian.mueller@th-nuernberg.de organization: Technische Hochschule Nürnberg, Kesslerplatz 12, 90489 Nuremberg, Germany – sequence: 2 givenname: Christian surname: Deuerlein fullname: Deuerlein, Christian organization: Technische Hochschule Nürnberg, Kesslerplatz 12, 90489 Nuremberg, Germany – sequence: 3 givenname: Michael surname: Koch fullname: Koch, Michael organization: Technische Hochschule Nürnberg, Kesslerplatz 12, 90489 Nuremberg, Germany |
| BookMark | eNqFkE1LxDAQhoOs4LruP_DQP9Caj6YNCoIsfsGCFz2HNJ1oym5SJkHov7fLehAPOpeZy_My73NOFiEGIOSS0YpR1lwN1YjReqw45ayisqItOyFLzhkvFW_Z4sd9RtYpDXSetqaC8SW52UwdYDl-TMlbsyvTlDLsCxexQBgREoTsw3thCoxdzAWEvgDnwOaIF-TUmV2C9fdekbeH-9fNU7l9eXze3G1LKyTPZaeMArCq5W1TG8aFaKXqKG95N__FeuAKjIVOGKG46J1TjZVgmZFKysbUYkXqY67FmBKC0yP6vcFJM6oPDvSgjw70wYGmUs8OZuz6F2Z9NtnHkNH43X_w7RGGudinB9TJeggWeo9zed1H_3fAF9KWfIo |
| CitedBy_id | crossref_primary_10_1016_j_rcim_2022_102515 crossref_primary_10_1007_s00170_025_15828_w crossref_primary_10_1007_s40436_021_00375_w crossref_primary_10_1007_s00170_023_11246_y crossref_primary_10_3390_jcp1030026 crossref_primary_10_1007_s40974_024_00327_7 crossref_primary_10_1109_TNSRE_2021_3120795 crossref_primary_10_3390_robotics13030035 crossref_primary_10_1016_j_artmed_2023_102692 |
| Cites_doi | 10.1109/TEVC.2018.2867601 10.1007/978-3-319-32552-1 10.3139/9783446460607.fm 10.1109/IROS.2015.7353615 10.1007/s00502-019-00741-4 10.1016/j.procir.2015.12.036 10.1016/j.procir.2019.02.023 10.1109/ROBIO49542.2019.8961854 10.1109/ISR.2013.6695708 10.1016/j.rcim.2020.102035 |
| ContentType | Journal Article |
| Copyright | 2021 The Author(s) |
| Copyright_xml | – notice: 2021 The Author(s) |
| DBID | 6I. AAFTH AAYXX CITATION |
| DOI | 10.1016/j.procir.2021.05.071 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2212-8271 |
| EndPage | 312 |
| ExternalDocumentID | 10_1016_j_procir_2021_05_071 S2212827121005369 |
| GroupedDBID | 0R~ 0SF 4.4 457 6I. AACTN AAEDT AAEDW AAFTH AAIKJ AALRI AAXUO ABMAC ACGFS ADBBV ADEZE AEXQZ AFTJW AGHFR AITUG ALMA_UNASSIGNED_HOLDINGS AMRAJ EBS EJD FDB HZ~ IXB KQ8 M41 M~E NCXOZ O-L O9- OK1 RIG ROL SSZ AAYWO AAYXX ACVFH ADCNI ADVLN AEUPX AFPUW AIGII AKBMS AKRWK AKYEP CITATION |
| ID | FETCH-LOGICAL-c352t-b8a8eec872764a1233758b0272b2121de28eaceb3a3823dff86c5ec1a58556a43 |
| ISICitedReferencesCount | 10 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001491622400052&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2212-8271 |
| IngestDate | Wed Nov 05 20:51:22 EST 2025 Tue Nov 18 22:11:18 EST 2025 Wed May 17 00:08:32 EDT 2023 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | programming by demonstration, digital twin augmented reality industrial robot cyber physical system |
| Language | English |
| License | This is an open access article under the CC BY-NC-ND license. |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c352t-b8a8eec872764a1233758b0272b2121de28eaceb3a3823dff86c5ec1a58556a43 |
| OpenAccessLink | https://dx.doi.org/10.1016/j.procir.2021.05.071 |
| PageCount | 6 |
| ParticipantIDs | crossref_primary_10_1016_j_procir_2021_05_071 crossref_citationtrail_10_1016_j_procir_2021_05_071 elsevier_sciencedirect_doi_10_1016_j_procir_2021_05_071 |
| PublicationCentury | 2000 |
| PublicationDate | 2021 2021-00-00 |
| PublicationDateYYYYMMDD | 2021-01-01 |
| PublicationDate_xml | – year: 2021 text: 2021 |
| PublicationDecade | 2020 |
| PublicationTitle | Procedia CIRP |
| PublicationYear | 2021 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Liu, Zhang (bib0001) 2015 W. Weber, Industrieroboter: Methoden der Steuerung und Regelung, 4th ed., 2019. Y. Mollard, T. Munzer, A. Baisero, M. Toussaint, M. Lopes, Robot programming from demonstration, feedback and transfer, in: W. Burgard (Ed.), 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015): Hamburg, Germany, 28 September-2 October 2015, IEEE, Piscataway, NJ, 2015, pp. 1825–1831. Akkaladevi, Pichler, Plasch, Ikeda, Hofmann (bib00011) 2019; 136 Amorim, Guimares, Mendona, Neto, Costa, Moreira (bib0008) 2021; 67 Milgram, Takemura, Utsumi, Kishino (bib00015) 1995 Stanton, C., Bogdanovych, A., Ratanasena, E., Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning, in: Proc. Australasian Conference on Robotics and Automation, 2012, pp. 51–59. Stadnicka, Litwin, Antonelli (bib00017) 2019; 79 . C. Piechnick, K. Wagner, Patent “Method, system and nonvolatile storage medium" (WO2020178435A1) INV. B25J9/16 G05B19/427, 2020 Hesse, Malisa (bib0003) 2016 Aßmann, Piechnick, Püschel, Piechnick, Falkenberg, Werner (bib00012) 2018 RoboDK, What is an End Effector and How Do You Use One?, 2020 Siciliano, Khatib (bib0002) 2016 J.-G. Ge, Programming by demonstration by optical tracking system for dual arm robot, in: IEEE ISR 2013, IEEE, 2013, pp. 1–7. Antonelli, Astanin (bib00013) 2016; 41 Z. Cao, H. Hu, Z. Zhao, Y. Lou, Robot Programming by Demonstration with Local Human Correction for Assembly, in: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), IEEE, 06.12.2019 -08.12.2019, pp. 166–171. Starke, Hendrich, Zhang (bib00018) 2019; 23 National Science Foundation, Cyber-Physical Systems (CPS) (nsf19553) | NSF-National Science Foundation (2019). Antonelli (10.1016/j.procir.2021.05.071_bib00013) 2016; 41 Akkaladevi (10.1016/j.procir.2021.05.071_bib00011) 2019; 136 Hesse (10.1016/j.procir.2021.05.071_bib0003) 2016 Amorim (10.1016/j.procir.2021.05.071_bib0008) 2021; 67 Starke (10.1016/j.procir.2021.05.071_bib00018) 2019; 23 Stadnicka (10.1016/j.procir.2021.05.071_bib00017) 2019; 79 10.1016/j.procir.2021.05.071_bib0004 10.1016/j.procir.2021.05.071_bib0005 10.1016/j.procir.2021.05.071_bib0006 Liu (10.1016/j.procir.2021.05.071_bib0001) 2015 10.1016/j.procir.2021.05.071_bib0007 10.1016/j.procir.2021.05.071_bib0009 10.1016/j.procir.2021.05.071_bib00010 Milgram (10.1016/j.procir.2021.05.071_bib00015) 1995 Siciliano (10.1016/j.procir.2021.05.071_bib0002) 2016 Aßmann (10.1016/j.procir.2021.05.071_bib00012) 2018 10.1016/j.procir.2021.05.071_bib00014 10.1016/j.procir.2021.05.071_bib00016 |
| References_xml | – reference: Y. Mollard, T. Munzer, A. Baisero, M. Toussaint, M. Lopes, Robot programming from demonstration, feedback and transfer, in: W. Burgard (Ed.), 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015): Hamburg, Germany, 28 September-2 October 2015, IEEE, Piscataway, NJ, 2015, pp. 1825–1831. – year: 2016 ident: bib0003 publication-title: Taschenbuch Robotik, Montage, Handhabung: Mit 34 Tabellen – reference: Stanton, C., Bogdanovych, A., Ratanasena, E., Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning, in: Proc. Australasian Conference on Robotics and Automation, 2012, pp. 51–59. – start-page: 484 year: 2018 end-page: 506 ident: bib00012 article-title: Modelling the World of a Smart Room for Robotic Co-working publication-title: Model-Driven Engineering and Software Development – volume: 41 start-page: 352 year: 2016 end-page: 357 ident: bib00013 article-title: Qualification of a Collaborative Human-robot Welding Cell publication-title: Procedia CIRP – volume: 23 start-page: 406 year: 2019 end-page: 420 ident: bib00018 article-title: Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation publication-title: IEEE Transactions on Evolutionary Computation – reference: W. Weber, Industrieroboter: Methoden der Steuerung und Regelung, 4th ed., 2019. – reference: . – reference: J.-G. Ge, Programming by demonstration by optical tracking system for dual arm robot, in: IEEE ISR 2013, IEEE, 2013, pp. 1–7. – reference: National Science Foundation, Cyber-Physical Systems (CPS) (nsf19553) | NSF-National Science Foundation (2019). – start-page: 769 year: 2015 end-page: 774 ident: bib0001 article-title: Toward welding robot with human knowledge: A remotely-controlled approach publication-title: IEEE Transactions on Automation Science and Engineering: A publication of the IEEE Robotics and Automation Society – volume: 136 start-page: 326 year: 2019 end-page: 333 ident: bib00011 article-title: Skill-based programming of complex robotic assembly tasks for industrial application [Skill-basierte Programmierung von komplexen Roboter-Montageaufgaben für die industrielle Applikation] publication-title: Elektrotechnik und Informationstechnik – reference: C. Piechnick, K. Wagner, Patent “Method, system and nonvolatile storage medium" (WO2020178435A1) INV. B25J9/16 G05B19/427, 2020, – reference: Z. Cao, H. Hu, Z. Zhao, Y. Lou, Robot Programming by Demonstration with Local Human Correction for Assembly, in: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), IEEE, 06.12.2019 -08.12.2019, pp. 166–171. – volume: 79 start-page: 718 year: 2019 end-page: 723 ident: bib00017 article-title: Human factor in intelligent manufacturing systems-knowledge acquisition and motivation publication-title: Procedia CIRP – volume: 67 start-page: 102035 year: 2021 ident: bib0008 article-title: Robust human position estimation in cooperative robotic cells publication-title: Robotics and Computer-Integrated Manufacturing – start-page: 282 year: 1995 end-page: 292 ident: bib00015 article-title: Augmented reality: a class of displays on the reality-virtuality continuum publication-title: International Society for Optics and Photonics – reference: RoboDK, What is an End Effector and How Do You Use One?, 2020, – year: 2016 ident: bib0002 publication-title: Springer Handbook of Robotics – ident: 10.1016/j.procir.2021.05.071_bib00014 – volume: 23 start-page: 406 year: 2019 ident: 10.1016/j.procir.2021.05.071_bib00018 article-title: Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation publication-title: IEEE Transactions on Evolutionary Computation doi: 10.1109/TEVC.2018.2867601 – year: 2016 ident: 10.1016/j.procir.2021.05.071_bib0002 doi: 10.1007/978-3-319-32552-1 – ident: 10.1016/j.procir.2021.05.071_bib0004 doi: 10.3139/9783446460607.fm – ident: 10.1016/j.procir.2021.05.071_bib00010 – ident: 10.1016/j.procir.2021.05.071_bib0009 doi: 10.1109/IROS.2015.7353615 – start-page: 769 year: 2015 ident: 10.1016/j.procir.2021.05.071_bib0001 article-title: Toward welding robot with human knowledge: A remotely-controlled approach – start-page: 484 year: 2018 ident: 10.1016/j.procir.2021.05.071_bib00012 article-title: Modelling the World of a Smart Room for Robotic Co-working – volume: 136 start-page: 326 year: 2019 ident: 10.1016/j.procir.2021.05.071_bib00011 article-title: Skill-based programming of complex robotic assembly tasks for industrial application [Skill-basierte Programmierung von komplexen Roboter-Montageaufgaben für die industrielle Applikation] publication-title: Elektrotechnik und Informationstechnik doi: 10.1007/s00502-019-00741-4 – volume: 41 start-page: 352 year: 2016 ident: 10.1016/j.procir.2021.05.071_bib00013 article-title: Qualification of a Collaborative Human-robot Welding Cell publication-title: Procedia CIRP doi: 10.1016/j.procir.2015.12.036 – volume: 79 start-page: 718 year: 2019 ident: 10.1016/j.procir.2021.05.071_bib00017 article-title: Human factor in intelligent manufacturing systems-knowledge acquisition and motivation publication-title: Procedia CIRP doi: 10.1016/j.procir.2019.02.023 – ident: 10.1016/j.procir.2021.05.071_bib0005 doi: 10.1109/ROBIO49542.2019.8961854 – ident: 10.1016/j.procir.2021.05.071_bib0006 doi: 10.1109/ISR.2013.6695708 – ident: 10.1016/j.procir.2021.05.071_bib0007 – year: 2016 ident: 10.1016/j.procir.2021.05.071_bib0003 – start-page: 282 year: 1995 ident: 10.1016/j.procir.2021.05.071_bib00015 article-title: Augmented reality: a class of displays on the reality-virtuality continuum publication-title: International Society for Optics and Photonics – ident: 10.1016/j.procir.2021.05.071_bib00016 – volume: 67 start-page: 102035 year: 2021 ident: 10.1016/j.procir.2021.05.071_bib0008 article-title: Robust human position estimation in cooperative robotic cells publication-title: Robotics and Computer-Integrated Manufacturing doi: 10.1016/j.rcim.2020.102035 |
| SSID | ssj0000740312 |
| Score | 2.1988397 |
| Snippet | Programming by Demonstration (PbD) is a method to program robots through the performance of a task by humans. Most implementations are online methods that use... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 307 |
| SubjectTerms | augmented reality cyber physical system industrial robot programming by demonstration, digital twin |
| Title | Cyber-physical-system for representing a robot end effector |
| URI | https://dx.doi.org/10.1016/j.procir.2021.05.071 |
| Volume | 100 |
| WOSCitedRecordID | wos001491622400052&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2212-8271 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000740312 issn: 2212-8271 databaseCode: M~E dateStart: 20120101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LTxsxELZK6IEeEBQQb-2BW-RqY-_DFqeSFoEoUdWmiNvKdrwSCO1Gm6QKF357x48NixJRQOKyyjrrdTIzGn8zngdCRzkPlU5yio31gCMuBBYJiXDMhSJEMJVbZ87Vj7TXY9fX_Kdvlzqy7QTSomDTKR--K6thDJhtUmdfwe7ZS2EAPgPT4Qpsh-uLGN-9l7rCQ09_7Eo122hCW8DSJBvZxMR2Vcpy3NbFwAd1lFUTqdoMAhCedvf81ywH7NKcq5906_TBUyEb0vVNT3R155tnuqIFjW8vStd0qhmn770N5NHPMJ8AY3QUgZ0PM-K6qHzRC8ZqJRuGDTVJXadbv-NSF0g9p8ydX-HWbCXqxtRuJR1bZbVe7UmZ7N9mWbMq2LCgWBK-hJZJCuaSCed8ePS7AWQCPWYOl2a_s86otGF_84stRiwNFNJfQ6vefAi-Oravow-6-Iw-NYpKbqDjhQIQgAAETQEIRGAFIAABCGoB2ER_Tr_3u2fYt8jACpDzGEsmmNaKAQpNIgEohIL9J0OSEgn_rjPQhMHOqiUV5rx3kOcsUbFWHQFWYpyIiG6hVlEWehsFHRUBXmJS6ZxFQkWcsFhQojiVAFsHfAfRmgyZ8vXjTRuTu6wOFLzNHPEyQ7wsjDMg3g7Cs1lDVz_lP8-nNYUzjwEdtstAKp6dufvmmXtoxdw5x9o-ao2riT5AH9Xf8c2oOrTi8w8m7IEi |
| linkProvider | ISSN International Centre |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cyber-physical-system+for+representing+a+robot+end+effector&rft.jtitle=Procedia+CIRP&rft.au=M%C3%BCller%2C+Fabian&rft.au=Deuerlein%2C+Christian&rft.au=Koch%2C+Michael&rft.date=2021&rft.pub=Elsevier+B.V&rft.issn=2212-8271&rft.eissn=2212-8271&rft.volume=100&rft.spage=307&rft.epage=312&rft_id=info:doi/10.1016%2Fj.procir.2021.05.071&rft.externalDocID=S2212827121005369 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2212-8271&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2212-8271&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2212-8271&client=summon |