LapFormer: surgical tool detection in laparoscopic surgical video using transformer architecture
One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of surgical tools in laparoscopic surgery videos, called LapFormer. The novelty of LapFormer is to use a Transformer architecture, which is a feed-fo...
Uloženo v:
| Vydáno v: | Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization Ročník 9; číslo 3; s. 302 - 307 |
|---|---|
| Hlavní autor: | |
| Médium: | Journal Article |
| Jazyk: | angličtina japonština |
| Vydáno: |
Taylor & Francis
04.05.2021
Informa UK Limited |
| Témata: | |
| ISSN: | 2168-1163, 2168-1171 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of surgical tools in laparoscopic surgery videos, called LapFormer. The novelty of LapFormer is to use a Transformer architecture, which is a feed-forward neural network architecture with attention mechanism, growing in popularity for natural language processing, for analysing inter-frame correlation in videos instead of using recurrent neural network families. To the best of our knowledge, no methods using a Transformer architecture for analysing laparoscopic surgery videos have been proposed. We evaluate our method on a dataset called Cholec80, which contains 80 videos of cholecystectomy surgeries. We confirm that our proposed method outperforms the conventional methods such as single-frame analysis with convolutional neural networks or multiple frame analysis with recurrent neural networks by 20.3 and 17.3 points in macro-F1 score, respectively. We also conduct an ablation study on how hyper-parameters for Transformer block in our proposed method affect the performance of the detection. |
|---|---|
| AbstractList | One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of surgical tools in laparoscopic surgery videos, called LapFormer. The novelty of LapFormer is to use a Transformer architecture, which is a feed-forward neural network architecture with attention mechanism, growing in popularity for natural language processing, for analysing inter-frame correlation in videos instead of using recurrent neural network families. To the best of our knowledge, no methods using a Transformer architecture for analysing laparoscopic surgery videos have been proposed. We evaluate our method on a dataset called Cholec80, which contains 80 videos of cholecystectomy surgeries. We confirm that our proposed method outperforms the conventional methods such as single-frame analysis with convolutional neural networks or multiple frame analysis with recurrent neural networks by 20.3 and 17.3 points in macro-F1 score, respectively. We also conduct an ablation study on how hyper-parameters for Transformer block in our proposed method affect the performance of the detection. |
| Author | Kondo, Satoshi |
| Author_xml | – sequence: 1 givenname: Satoshi surname: Kondo fullname: Kondo, Satoshi email: satoshi.kondo@konicaminolta.com organization: Konica Minolta, Inc |
| BackLink | https://cir.nii.ac.jp/crid/1871991017630640896$$DView record in CiNii |
| BookMark | eNqFkEFPwyAYhomZiXPuJ5hw8NoJpVDQi2ZxarLEi56RAp2YrjTQafbvpW5q4kE5AHnzPR98zzEYtb61AJxiNMOIo_McM44xI7Mc5SnihFKKDsB4yDOMSzz6vjNyBKYxvqK0OGOE0TF4Xqpu4cPahgsYN2HltGpg730Dje2t7p1voWthozoVfNS-c_qn7s0Z6-EmunYF-6DaWH92giroFzfQm2BPwGGtmmin-3MCnhY3j_O7bPlwez-_Xma6QLTPLM8LhEydq7LShbCozEWlLTe65KgUWleVqVMuKKmJMYYSTThGAleFULTiZALorq9O_4zB1rILbq3CVmIkB1Pyy5QcTMm9qcRd_uK069Uwd5rINf_SZzu6dS6Bw455iYXACJeMIFYgLlgqu9qVuXZQpN59aIzs1bbxoU7itIuS_P3SB15xkAI |
| CitedBy_id | crossref_primary_10_1002_rcs_70089 crossref_primary_10_1109_TPAMI_2023_3243465 crossref_primary_10_1080_21681163_2022_2145238 crossref_primary_10_1038_s43856_024_00581_0 crossref_primary_10_1002_rcs_2445 crossref_primary_10_3390_bioengineering9120737 crossref_primary_10_1109_TMI_2023_3335406 crossref_primary_10_1109_TMI_2023_3279838 crossref_primary_10_1007_s11548_022_02691_3 crossref_primary_10_1093_bjsopen_zraf073 crossref_primary_10_1109_TIM_2023_3298396 crossref_primary_10_1007_s13042_023_01875_w crossref_primary_10_1080_21681163_2022_2152371 crossref_primary_10_1007_s00371_025_04161_8 crossref_primary_10_1049_htl2_12060 crossref_primary_10_1109_TMI_2022_3177077 |
| Cites_doi | 10.1007/s11263-015-0816-y 10.1109/CVPR.2015.7298594 10.1016/j.media.2019.101572 10.1109/CBMI.2015.7153616 10.3115/v1/D14-1179 10.1016/j.ipm.2009.03.002 10.1109/CVPR.2019.00033 10.1109/CVPR.2016.90 10.1162/neco.1997.9.8.1735 10.1109/TMI.2016.2593957 |
| ContentType | Journal Article |
| Copyright | 2020 Informa UK Limited, trading as Taylor & Francis Group 2020 |
| Copyright_xml | – notice: 2020 Informa UK Limited, trading as Taylor & Francis Group 2020 |
| DBID | RYH AAYXX CITATION |
| DOI | 10.1080/21681163.2020.1835550 |
| DatabaseName | CiNii Complete CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2168-1171 |
| EndPage | 307 |
| ExternalDocumentID | 10_1080_21681163_2020_1835550 1835550 |
| Genre | Research Article |
| GroupedDBID | 0BK 30N 4.4 AAGDL AAJMT AALDU AAMIU AAPUL AAQRR ABLIJ ABPAQ ABXUL ABXYU ACGFS ADCVX ADGTB ADMLS AEISY AFRVT AGDLA AHDZW AIJEM AIYEW AKBVH AKOOK ALMA_UNASSIGNED_HOLDINGS ALQZU AQTUD ARCSS BLEHA CCCUG EBS EUPTU GTTXZ H13 HZ~ KYCEM LJTGL M4Z O9- RIG RNANH ROSJB RTWRZ SNACF SOJIQ TBQAZ TDBHL TEN TFL TFT TFW TTHFI TUROJ RYH AAYXX CITATION |
| ID | FETCH-LOGICAL-c405t-e82400df2a7bc49e0729bce8dc78079ccbbdf49e953f3ddd53c381091b49a5b83 |
| IEDL.DBID | TFW |
| ISICitedReferencesCount | 29 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000581900900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2168-1163 |
| IngestDate | Sat Nov 29 06:34:10 EST 2025 Tue Nov 18 21:09:46 EST 2025 Mon Nov 10 09:14:55 EST 2025 Mon Oct 20 23:47:46 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English Japanese |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c405t-e82400df2a7bc49e0729bce8dc78079ccbbdf49e953f3ddd53c381091b49a5b83 |
| ORCID | 0000-0002-4941-4920 |
| OpenAccessLink | https://cir.nii.ac.jp/crid/1871991017630640896 |
| PageCount | 6 |
| ParticipantIDs | nii_cinii_1871991017630640896 crossref_primary_10_1080_21681163_2020_1835550 informaworld_taylorfrancis_310_1080_21681163_2020_1835550 crossref_citationtrail_10_1080_21681163_2020_1835550 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-05-04 |
| PublicationDateYYYYMMDD | 2021-05-04 |
| PublicationDate_xml | – month: 05 year: 2021 text: 2021-05-04 day: 04 |
| PublicationDecade | 2020 |
| PublicationTitle | Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization |
| PublicationYear | 2021 |
| Publisher | Taylor & Francis Informa UK Limited |
| Publisher_xml | – name: Taylor & Francis – name: Informa UK Limited |
| References | cit0011 cit0012 cit0010 Kitaev N (cit0007) 2020 Ba JL (cit0001) 2016; 1607 Namazi B (cit0009) 2019; 1905 cit0008 cit0006 cit0004 cit0015 cit0005 cit0016 cit0002 cit0013 cit0003 cit0014 |
| References_xml | – ident: cit0011 doi: 10.1007/s11263-015-0816-y – ident: cit0013 doi: 10.1109/CVPR.2015.7298594 – ident: cit0006 doi: 10.1016/j.media.2019.101572 – volume-title: International Conference on Learning Representaitons (ICLR), Virtual Conference, Formerly Addis Ababa, Ethiopia year: 2020 ident: cit0007 – ident: cit0010 doi: 10.1109/CBMI.2015.7153616 – ident: cit0002 doi: 10.3115/v1/D14-1179 – ident: cit0008 – ident: cit0015 – ident: cit0016 – ident: cit0012 doi: 10.1016/j.ipm.2009.03.002 – ident: cit0003 doi: 10.1109/CVPR.2019.00033 – ident: cit0004 doi: 10.1109/CVPR.2016.90 – ident: cit0005 doi: 10.1162/neco.1997.9.8.1735 – ident: cit0014 doi: 10.1109/TMI.2016.2593957 – volume: 1905 start-page: 08983 year: 2019 ident: cit0009 publication-title: arXiv – volume: 1607 start-page: 06450 year: 2016 ident: cit0001 publication-title: arXiv |
| SSID | ssj0000866365 ssib044168314 ssib039557987 ssib024195514 |
| Score | 2.317325 |
| Snippet | One of the most essential steps in the surgical workflow analysis is recognition of surgical tool presence. We propose a method to detect the presence of... |
| SourceID | crossref nii informaworld |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 302 |
| SubjectTerms | Laparoscopy surgical workflow analysis transformer |
| Title | LapFormer: surgical tool detection in laparoscopic surgical video using transformer architecture |
| URI | https://www.tandfonline.com/doi/abs/10.1080/21681163.2020.1835550 https://cir.nii.ac.jp/crid/1871991017630640896 |
| Volume | 9 |
| WOSCitedRecordID | wos000581900900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAWR databaseName: Taylor & Francis Journals Complete customDbUrl: eissn: 2168-1171 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000866365 issn: 2168-1163 databaseCode: TFW dateStart: 20130301 isFulltext: true titleUrlDefault: https://www.tandfonline.com providerName: Taylor & Francis |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwELYQYoCBN6JAkQfWQBI7ic2GEBUDqhiK6Bb8CopUJVWa8vu5S5PSDqgDLB6snHW6XHzfOefvCLlhlisLGnqxb2OPK2E8qXTmGalMpBXPEqubZhPJcCjGY_naVhPO2rJKzKGzBVFEs1fjx630rKuIuwuDWASAIyC7C2EKMETUZO2A7NHHR4P35SkLAPaYNf0kUchDqe4az28LrQWoNfpSCD9Fnq-En8HBPyh-SPZb7EkfFs5yRLZccUz2VhgJT8jHi5oOQBVX3dPZvGq2RVqX5YRaVzdVWwXNCzqBEIs0mOU0Nz_P4ZW-kmIl_SetO0DsKrr6s-KUvA2eRo_PXtuEwTOA5WrPCawytVmoEm24dMg0ro0T1iTCT6QxWtsM5mXEMmatjZhB0jAZaC5VpAU7I9tFWbhzQgVgFc2Eb0OksGGwoglsYoXPVQQD7xHeWT41LUM5NsqYpEFLZNoZMEUDpq0Be-R2KTZdUHRsEpCrrzWtm7ORbNHIJGUbZPvgA6AejgFkm4CvYVOLMZHzhYwv_rD2JdkNsWIGyyn5Fdmuq7nrkx3zVeez6rpx7G-SSvGw |
| linkProvider | Taylor & Francis |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT4QwEG6MmqgH38a3PXhFgRZovRnjRuO6pzV6w74wJBvYIPr7nWFB8WA86KWHhmkmpXS-KV-_IeSUWa4seOjFvo09roTxpNKZZ6QykVY8S6xuik0ko5F4epL9uzBIq8QcOpsJRTR7NX7ceBjdUeLOwyAWAQAJSO9C6AIQEWHavhBBrEVa33jw-HnOApA9Zk1FSbTy0Ky7yPPTSN9C1DcBUwhARZ73AtBg7T9cXyerLfykl7P1skHmXLFJVnqihFvkeaimA_DFVRf09a1qdkZal-WEWlc3xK2C5gWdQJRFJcxympuv5_BWX0mRTP9C6w4Tu4r2_1dsk4fB9fjqxmvrMHgG4FztOYFEU5uFKtGGS4di49o4YU0i_EQao7XNoF9GLGPW2ogZ1A2TgeZSRVqwHTJflIXbJVQAXNFM-DZEFRsGI5rAJlb4XEXQ8D3Cu6lPTStSjrUyJmnQapl2E5jiBKbtBO6Rs0-z6Uyl4zcD2X-vad0cj2SzWiYp-8X2CBYBuIdtAAknQGzY12LM5Xwh4_0_jH1Clm7G98N0eDu6OyDLIRJokF3JD8l8Xb25I7Jo3uv8tTpuVvkH3FP10Q |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT4QwEG6MGqMH38a3PXhFgRZovRmVaDQbDxq9YV8Ykg1sWPT3O8OCrgfjQS89NEwzGYbOTPn6DSHHzHJlQUMv9m3scSWMJ5XOPSOVibTieWJ122wiGQzE87O879CE4w5WiTV0PiGKaPdq_LhHNu8RcadhEIsA8gio7kKYghwiwqp9riXHApd-SJ8-j1kgY49Z21ASpTwU6-_x_LTStwj1jb8U4k9ZFFPxJ135B81XyXKXfNLzibeskRlXrpOlKUrCDfJyp0YpqOLqMzp-q9t9kTZVNaTWNS1sq6RFSYcQY5EHsxoV5us5vNNXUYTSv9Kmz4hdTaf_VmySx_Tq4eLa67oweAaSucZzAmGmNg9Vog2XDqnGtXHCmkT4iTRGa5vDvIxYzqy1ETPIGiYDzaWKtGBbZLasSrdNqIBkRTPh2xA5bBisaAKbWOFzFcHAdwjvLZ-ZjqIcO2UMs6BjMu0NmKEBs86AO-TkU2w04ej4TUBOv9asaQ9H8kknk4z9InsAPgDq4RhAuQkJNuxqMVZyvpDx7h_WPiIL95dpdnczuN0jiyGiZxBayffJbFO_uQMyb96bYlwftj7-Acdk9HU |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=LapFormer%3A+surgical+tool+detection+in+laparoscopic+surgical+video+using+transformer+architecture&rft.jtitle=Computer+methods+in+biomechanics+and+biomedical+engineering.&rft.au=Kondo%2C+Satoshi&rft.date=2021-05-04&rft.issn=2168-1163&rft.eissn=2168-1171&rft.volume=9&rft.issue=3&rft.spage=302&rft.epage=307&rft_id=info:doi/10.1080%2F21681163.2020.1835550&rft.externalDBID=n%2Fa&rft.externalDocID=10_1080_21681163_2020_1835550 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2168-1163&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2168-1163&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2168-1163&client=summon |