LapFormer: surgical tool detection in laparoscopic surgical video using transformer architecture
One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of surgical tools in laparoscopic surgery videos, called LapFormer. The novelty of LapFormer is to use a Transformer architecture, which is a feed-fo...
Gespeichert in:
| Veröffentlicht in: | Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization Jg. 9; H. 3; S. 302 - 307 |
|---|---|
| 1. Verfasser: | |
| Format: | Journal Article |
| Sprache: | Englisch Japanisch |
| Veröffentlicht: |
Taylor & Francis
04.05.2021
Informa UK Limited |
| Schlagworte: | |
| ISSN: | 2168-1163, 2168-1171 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of surgical tools in laparoscopic surgery videos, called LapFormer. The novelty of LapFormer is to use a Transformer architecture, which is a feed-forward neural network architecture with attention mechanism, growing in popularity for natural language processing, for analysing inter-frame correlation in videos instead of using recurrent neural network families. To the best of our knowledge, no methods using a Transformer architecture for analysing laparoscopic surgery videos have been proposed. We evaluate our method on a dataset called Cholec80, which contains 80 videos of cholecystectomy surgeries. We confirm that our proposed method outperforms the conventional methods such as single-frame analysis with convolutional neural networks or multiple frame analysis with recurrent neural networks by 20.3 and 17.3 points in macro-F1 score, respectively. We also conduct an ablation study on how hyper-parameters for Transformer block in our proposed method affect the performance of the detection. |
|---|---|
| AbstractList | One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of surgical tools in laparoscopic surgery videos, called LapFormer. The novelty of LapFormer is to use a Transformer architecture, which is a feed-forward neural network architecture with attention mechanism, growing in popularity for natural language processing, for analysing inter-frame correlation in videos instead of using recurrent neural network families. To the best of our knowledge, no methods using a Transformer architecture for analysing laparoscopic surgery videos have been proposed. We evaluate our method on a dataset called Cholec80, which contains 80 videos of cholecystectomy surgeries. We confirm that our proposed method outperforms the conventional methods such as single-frame analysis with convolutional neural networks or multiple frame analysis with recurrent neural networks by 20.3 and 17.3 points in macro-F1 score, respectively. We also conduct an ablation study on how hyper-parameters for Transformer block in our proposed method affect the performance of the detection. |
| Author | Kondo, Satoshi |
| Author_xml | – sequence: 1 givenname: Satoshi surname: Kondo fullname: Kondo, Satoshi email: satoshi.kondo@konicaminolta.com organization: Konica Minolta, Inc |
| BackLink | https://cir.nii.ac.jp/crid/1871991017630640896$$DView record in CiNii |
| BookMark | eNqFkEFPwyAYhomZiXPuJ5hw8NoJpVDQi2ZxarLEi56RAp2YrjTQafbvpW5q4kE5AHnzPR98zzEYtb61AJxiNMOIo_McM44xI7Mc5SnihFKKDsB4yDOMSzz6vjNyBKYxvqK0OGOE0TF4Xqpu4cPahgsYN2HltGpg730Dje2t7p1voWthozoVfNS-c_qn7s0Z6-EmunYF-6DaWH92giroFzfQm2BPwGGtmmin-3MCnhY3j_O7bPlwez-_Xma6QLTPLM8LhEydq7LShbCozEWlLTe65KgUWleVqVMuKKmJMYYSTThGAleFULTiZALorq9O_4zB1rILbq3CVmIkB1Pyy5QcTMm9qcRd_uK069Uwd5rINf_SZzu6dS6Bw455iYXACJeMIFYgLlgqu9qVuXZQpN59aIzs1bbxoU7itIuS_P3SB15xkAI |
| CitedBy_id | crossref_primary_10_1002_rcs_70089 crossref_primary_10_1109_TPAMI_2023_3243465 crossref_primary_10_1080_21681163_2022_2145238 crossref_primary_10_1038_s43856_024_00581_0 crossref_primary_10_1002_rcs_2445 crossref_primary_10_3390_bioengineering9120737 crossref_primary_10_1109_TMI_2023_3335406 crossref_primary_10_1109_TMI_2023_3279838 crossref_primary_10_1007_s11548_022_02691_3 crossref_primary_10_1093_bjsopen_zraf073 crossref_primary_10_1109_TIM_2023_3298396 crossref_primary_10_1007_s13042_023_01875_w crossref_primary_10_1080_21681163_2022_2152371 crossref_primary_10_1007_s00371_025_04161_8 crossref_primary_10_1049_htl2_12060 crossref_primary_10_1109_TMI_2022_3177077 |
| Cites_doi | 10.1007/s11263-015-0816-y 10.1109/CVPR.2015.7298594 10.1016/j.media.2019.101572 10.1109/CBMI.2015.7153616 10.3115/v1/D14-1179 10.1016/j.ipm.2009.03.002 10.1109/CVPR.2019.00033 10.1109/CVPR.2016.90 10.1162/neco.1997.9.8.1735 10.1109/TMI.2016.2593957 |
| ContentType | Journal Article |
| Copyright | 2020 Informa UK Limited, trading as Taylor & Francis Group 2020 |
| Copyright_xml | – notice: 2020 Informa UK Limited, trading as Taylor & Francis Group 2020 |
| DBID | RYH AAYXX CITATION |
| DOI | 10.1080/21681163.2020.1835550 |
| DatabaseName | CiNii Complete CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2168-1171 |
| EndPage | 307 |
| ExternalDocumentID | 10_1080_21681163_2020_1835550 1835550 |
| Genre | Research Article |
| GroupedDBID | 0BK 30N 4.4 AAGDL AAJMT AALDU AAMIU AAPUL AAQRR ABLIJ ABPAQ ABXUL ABXYU ACGFS ADCVX ADGTB ADMLS AEISY AFRVT AGDLA AHDZW AIJEM AIYEW AKBVH AKOOK ALMA_UNASSIGNED_HOLDINGS ALQZU AQTUD ARCSS BLEHA CCCUG EBS EUPTU GTTXZ H13 HZ~ KYCEM LJTGL M4Z O9- RIG RNANH ROSJB RTWRZ SNACF SOJIQ TBQAZ TDBHL TEN TFL TFT TFW TTHFI TUROJ RYH AAYXX CITATION |
| ID | FETCH-LOGICAL-c405t-e82400df2a7bc49e0729bce8dc78079ccbbdf49e953f3ddd53c381091b49a5b83 |
| IEDL.DBID | TFW |
| ISICitedReferencesCount | 29 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000581900900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2168-1163 |
| IngestDate | Sat Nov 29 06:34:10 EST 2025 Tue Nov 18 21:09:46 EST 2025 Mon Nov 10 09:14:55 EST 2025 Mon Oct 20 23:47:46 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English Japanese |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c405t-e82400df2a7bc49e0729bce8dc78079ccbbdf49e953f3ddd53c381091b49a5b83 |
| ORCID | 0000-0002-4941-4920 |
| OpenAccessLink | https://cir.nii.ac.jp/crid/1871991017630640896 |
| PageCount | 6 |
| ParticipantIDs | nii_cinii_1871991017630640896 crossref_primary_10_1080_21681163_2020_1835550 informaworld_taylorfrancis_310_1080_21681163_2020_1835550 crossref_citationtrail_10_1080_21681163_2020_1835550 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-05-04 |
| PublicationDateYYYYMMDD | 2021-05-04 |
| PublicationDate_xml | – month: 05 year: 2021 text: 2021-05-04 day: 04 |
| PublicationDecade | 2020 |
| PublicationTitle | Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization |
| PublicationYear | 2021 |
| Publisher | Taylor & Francis Informa UK Limited |
| Publisher_xml | – name: Taylor & Francis – name: Informa UK Limited |
| References | cit0011 cit0012 cit0010 Kitaev N (cit0007) 2020 Ba JL (cit0001) 2016; 1607 Namazi B (cit0009) 2019; 1905 cit0008 cit0006 cit0004 cit0015 cit0005 cit0016 cit0002 cit0013 cit0003 cit0014 |
| References_xml | – ident: cit0011 doi: 10.1007/s11263-015-0816-y – ident: cit0013 doi: 10.1109/CVPR.2015.7298594 – ident: cit0006 doi: 10.1016/j.media.2019.101572 – volume-title: International Conference on Learning Representaitons (ICLR), Virtual Conference, Formerly Addis Ababa, Ethiopia year: 2020 ident: cit0007 – ident: cit0010 doi: 10.1109/CBMI.2015.7153616 – ident: cit0002 doi: 10.3115/v1/D14-1179 – ident: cit0008 – ident: cit0015 – ident: cit0016 – ident: cit0012 doi: 10.1016/j.ipm.2009.03.002 – ident: cit0003 doi: 10.1109/CVPR.2019.00033 – ident: cit0004 doi: 10.1109/CVPR.2016.90 – ident: cit0005 doi: 10.1162/neco.1997.9.8.1735 – ident: cit0014 doi: 10.1109/TMI.2016.2593957 – volume: 1905 start-page: 08983 year: 2019 ident: cit0009 publication-title: arXiv – volume: 1607 start-page: 06450 year: 2016 ident: cit0001 publication-title: arXiv |
| SSID | ssj0000866365 ssib044168314 ssib039557987 ssib024195514 |
| Score | 2.317325 |
| Snippet | One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of... |
| SourceID | crossref nii informaworld |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 302 |
| SubjectTerms | Laparoscopy surgical workflow analysis transformer |
| Title | LapFormer: surgical tool detection in laparoscopic surgical video using transformer architecture |
| URI | https://www.tandfonline.com/doi/abs/10.1080/21681163.2020.1835550 https://cir.nii.ac.jp/crid/1871991017630640896 |
| Volume | 9 |
| WOSCitedRecordID | wos000581900900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAWR databaseName: Taylor & Francis Journals Complete customDbUrl: eissn: 2168-1171 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000866365 issn: 2168-1163 databaseCode: TFW dateStart: 20130301 isFulltext: true titleUrlDefault: https://www.tandfonline.com providerName: Taylor & Francis |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwELZQxQADb0SBIg-sgSTOw2ZDiIihqhiK6BbiR1CkKqmSlN_PXR6lHVAHWDJYOcu6OHffl5y_I-RWC9cVQcItBuQDCArzIQ4GjqUDm-lUKJ3y5qDwOJxM-GwmXrtqwqorq0QOnbZCEU2sxpc7kVVfEXfvOgF3AEcAu3NhCDCE37B2SP3Yw2Aava--sgBgD1jTTxKNLLTqj_H8NtFGgtqQL4X0k2fZWvqJDv9h4UfkoMOe9LHdLMdkx-QnZH9NkfCUfIyTRQRLMeUDrZZlExZpXRRzqk3dVG3lNMvpHFIsymAWi0z93IdH-gqKlfSftO4BsSnp-s-KM_IWPU-fXqyuCYOlAMvVluFYZapTNwml8oRBpXGpDNcq5HYolJJSpzAufJYyrbXPFIqGCUd6IvElZ-dkkBe5uSDUS2QaAOIQPDEeB-ZmJww4sgoZA2PlDInXez5WnUI5NsqYx04nZNo7MEYHxp0Dh-RuZbZoJTq2GYj1xxrXzbeRtG1kErMttiPYA7A8vDrANgFfQ1ALkMjZXASXf5j7iuy5WDGD5ZTeNRnU5dKMyK76qrOqvGk29jd0t_AU |
| linkProvider | Taylor & Francis |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT4NAEN6YaqIefBurVvfgFQWWx643YyQ11p5q7A1hdzEkDTSU-vudoVDpwfSgFw6E2WyGYWa-ZeYbQm6UsG3hRdxgAD4AoDAX_KBnGcozmUqEVAmvGoUH_nDIx2PR7oXBskrE0MmCKKLy1fhx42F0UxJ3Z1setyCRAHhnwy1IIlyE7ZsuxFq08lHwvjxngZTdY9VESZQyUKxp5PltpZUQtUJgCgEoS9NWAAr2_2PrB2SvTj_pw8JeDsmGzo7IbouU8Jh8DKJpAHvRxT2dzYvKM9IyzydU6bIq3MpomtEJRFlkwsynqfx5Drv6corF9J-0bHJiXdD2_4oT8hY8jR77Rj2HwZCQzpWG5lhoqhI78mPpCI1k47HUXEmfm76QMo5VAveFyxKmlHKZRN4wYcWOiNyYs1PSyfJMnxHqRHHiQdIheKQdDuDNjBjAZOkzBsLS6hKnUX0oa5JynJUxCa2ay7RRYIgKDGsFdsntUmy6YOlYJyDa7zUsq-ORZDHLJGRrZHtgBLA9vFoAOCHFBr_mIZYzufDO_7D2Ndnuj14H4eB5-HJBdmwsoMHqSueSdMpirntkS36V6ay4qqz8G8LG9D8 |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT4NAEN4YNUYPvo31uQevaGFh2fVmVKKxaXrQ2BvCPgxJAw2l_n5nKNR6MB70wmHDbDbDMvN9MPsNIRdaep7kiXAYkA8gKCyAOMhdR_Mu01YqbUV9ULgX9vtiOJSDpppw0pRVIoe2M6GIOlbjyz3Wtq2Iu_JcLlzAEcDuPBgCDBEga18B6MyRfz1Hr_PPLIDYOasbSqKVg2btOZ6fZvqWob7pl0L-ybNsIf9EW_-w8m2y2YBPejPbLTtkyeS7ZGNBknCPvPWScQRLMeU1nUzLOi7SqihGVJuqLtvKaZbTEeRY1MEsxpn6ug_P9BUUS-nfadUiYlPSxb8V--Qlun--fXCaLgyOAjBXOUZgmam2XhKmypcGpcZTZYRWoeiGUqk01RbGZcAs01oHTKFqmHRTXyZBKtgBWc6L3BwS6iep5QA5pEiML4C6dRMGJFmFjIGxcjvEbz0fq0aiHDtljGK3UTJtHRijA-PGgR1yOTcbzzQ6fjOQi481ruqPI3bWySRmv9iewh6A5eHVBboJABuiGkcm1xWSH_1h7nOyNriL4t5j_-mYrHtYPYOllf4JWa7KqTklq-qjyiblWb3HPwHZYvLw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=LapFormer%3A+surgical+tool+detection+in+laparoscopic+surgical+video+using+transformer+architecture&rft.jtitle=Computer+methods+in+biomechanics+and+biomedical+engineering.&rft.au=Kondo%2C+Satoshi&rft.date=2021-05-04&rft.pub=Taylor+%26+Francis&rft.issn=2168-1163&rft.eissn=2168-1171&rft.volume=9&rft.issue=3&rft.spage=302&rft.epage=307&rft_id=info:doi/10.1080%2F21681163.2020.1835550&rft.externalDocID=1835550 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2168-1163&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2168-1163&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2168-1163&client=summon |