In EDS ansehen

Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images

Gespeichert in:
Bibliographische Detailangaben
Titel: Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images
Autoren: Maquiling, Virmarie, Byrne, Sean Anthony, Niehorster, Diederick C., Carminati, Marco, Kasneci, Enkelejda
Weitere Verfasser: Lund University, Joint Faculties of Humanities and Theology, Units, Lund University Humanities Lab, Lunds universitet, Humanistiska och teologiska fakulteterna, Fakultetsgemensamma verksamheter, Humanistlaboratoriet, Originator, Lund University, Faculty of Social Sciences, Departments of Administrative, Economic and Social Sciences, Department of Psychology, Lunds universitet, Samhällsvetenskapliga fakulteten, Samhällsvetenskapliga institutioner och centrumbildningar, Institutionen för psykologi, Originator
Quelle: Proceedings of the ACM on Computer Graphics and Interactive Techniques. 8(2):1-16
Schlagwörter: Natural Sciences, Computer and Information Sciences, Naturvetenskap, Data- och informationsvetenskap (Datateknik), Natural Language Processing, Språkbehandling och datorlingvistik
Beschreibung: We explore the transformative potential of SAM 2, a vision foundation model, in advancing gaze estimation. SAM 2 addresses key challenges in gaze estimation by significantly reducing annotation time, simplifying deployment, and enhancing segmentation accuracy. Utilizing its zero-shot capabilities with minimal user input—a single click per video—we tested SAM 2 on over 14 million eye images from a diverse range of datasets, including the EDS challenge datasets and Labelled Pupils in the Wild. This is the first application of SAM 2 to the gaze estimation domain. Remarkably, SAM 2 matches the performance of domain-specific models in pupil segmentation, achieving competitive mIOU scores of up to 93% without fine-tuning. We argue that SAM 2 achieves the sought-after standard of domain generalization, with consistent mIOU scores (89.71%-93.74%) across diverse datasets, from virtual reality to "gaze-in-the-wild" scenarios. We provide our code and segmentation masks for these datasets to promote further research.
Zugangs-URL: https://doi.org/10.1145/3729409
Datenbank: SwePub
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://doi.org/10.1145/3729409#
    Name: EDS - SwePub (s4221598)
    Category: fullText
    Text: View record in SwePub
  – Url: https://resolver.ebscohost.com/openurl?sid=EBSCO:edsswe&genre=article&issn=25776193&ISBN=&volume=8&issue=2&date=20250601&spage=1&pages=1-16&title=Proceedings of the ACM on Computer Graphics and Interactive Techniques&atitle=Zero-Shot%20Pupil%20Segmentation%20with%20SAM%202%3A%20A%20Case%20Study%20of%20Over%2014%20Million%20Images&aulast=Maquiling%2C%20Virmarie&id=DOI:10.1145/3729409
    Name: Full Text Finder
    Category: fullText
    Text: Full Text Finder
    Icon: https://imageserver.ebscohost.com/branding/images/FTF.gif
    MouseOverText: Full Text Finder
  – Url: https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=EBSCO&SrcAuth=EBSCO&DestApp=WOS&ServiceName=TransferToWoS&DestLinkType=GeneralSearchSummary&Func=Links&author=Maquiling%20V
    Name: ISI
    Category: fullText
    Text: Nájsť tento článok vo Web of Science
    Icon: https://imagesrvr.epnet.com/ls/20docs.gif
    MouseOverText: Nájsť tento článok vo Web of Science
Header DbId: edsswe
DbLabel: SwePub
An: edsswe.oai.portal.research.lu.se.publications.8ff9fd0f.38de.4a82.bd02.500b02414d4a
RelevancyScore: 1133
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1132.50830078125
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Maquiling%2C+Virmarie%22">Maquiling, Virmarie</searchLink><br /><searchLink fieldCode="AR" term="%22Byrne%2C+Sean+Anthony%22">Byrne, Sean Anthony</searchLink><br /><searchLink fieldCode="AR" term="%22Niehorster%2C+Diederick+C%2E%22">Niehorster, Diederick C.</searchLink><br /><searchLink fieldCode="AR" term="%22Carminati%2C+Marco%22">Carminati, Marco</searchLink><br /><searchLink fieldCode="AR" term="%22Kasneci%2C+Enkelejda%22">Kasneci, Enkelejda</searchLink>
– Name: Author
  Label: Contributors
  Group: Au
  Data: Lund University, Joint Faculties of Humanities and Theology, Units, Lund University Humanities Lab, Lunds universitet, Humanistiska och teologiska fakulteterna, Fakultetsgemensamma verksamheter, Humanistlaboratoriet, Originator<br />Lund University, Faculty of Social Sciences, Departments of Administrative, Economic and Social Sciences, Department of Psychology, Lunds universitet, Samhällsvetenskapliga fakulteten, Samhällsvetenskapliga institutioner och centrumbildningar, Institutionen för psykologi, Originator
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <i>Proceedings of the ACM on Computer Graphics and Interactive Techniques</i>. 8(2):1-16
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Natural+Sciences%22">Natural Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+and+Information+Sciences%22">Computer and Information Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22Naturvetenskap%22">Naturvetenskap</searchLink><br /><searchLink fieldCode="DE" term="%22Data-+och+informationsvetenskap+%28Datateknik%29%22">Data- och informationsvetenskap (Datateknik)</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Språkbehandling+och+datorlingvistik%22">Språkbehandling och datorlingvistik</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: We explore the transformative potential of SAM 2, a vision foundation model, in advancing gaze estimation. SAM 2 addresses key challenges in gaze estimation by significantly reducing annotation time, simplifying deployment, and enhancing segmentation accuracy. Utilizing its zero-shot capabilities with minimal user input—a single click per video—we tested SAM 2 on over 14 million eye images from a diverse range of datasets, including the EDS challenge datasets and Labelled Pupils in the Wild. This is the first application of SAM 2 to the gaze estimation domain. Remarkably, SAM 2 matches the performance of domain-specific models in pupil segmentation, achieving competitive mIOU scores of up to 93% without fine-tuning. We argue that SAM 2 achieves the sought-after standard of domain generalization, with consistent mIOU scores (89.71%-93.74%) across diverse datasets, from virtual reality to "gaze-in-the-wild" scenarios. We provide our code and segmentation masks for these datasets to promote further research.
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://doi.org/10.1145/3729409" linkWindow="_blank">https://doi.org/10.1145/3729409</link>
PLink https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsswe&AN=edsswe.oai.portal.research.lu.se.publications.8ff9fd0f.38de.4a82.bd02.500b02414d4a
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1145/3729409
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 1
    Subjects:
      – SubjectFull: Natural Sciences
        Type: general
      – SubjectFull: Computer and Information Sciences
        Type: general
      – SubjectFull: Naturvetenskap
        Type: general
      – SubjectFull: Data- och informationsvetenskap (Datateknik)
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Språkbehandling och datorlingvistik
        Type: general
    Titles:
      – TitleFull: Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Maquiling, Virmarie
      – PersonEntity:
          Name:
            NameFull: Byrne, Sean Anthony
      – PersonEntity:
          Name:
            NameFull: Niehorster, Diederick C.
      – PersonEntity:
          Name:
            NameFull: Carminati, Marco
      – PersonEntity:
          Name:
            NameFull: Kasneci, Enkelejda
      – PersonEntity:
          Name:
            NameFull: Lund University, Joint Faculties of Humanities and Theology, Units, Lund University Humanities Lab, Lunds universitet, Humanistiska och teologiska fakulteterna, Fakultetsgemensamma verksamheter, Humanistlaboratoriet, Originator
      – PersonEntity:
          Name:
            NameFull: Lund University, Faculty of Social Sciences, Departments of Administrative, Economic and Social Sciences, Department of Psychology, Lunds universitet, Samhällsvetenskapliga fakulteten, Samhällsvetenskapliga institutioner och centrumbildningar, Institutionen för psykologi, Originator
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 25776193
            – Type: issn-locals
              Value: SWEPUB_FREE
            – Type: issn-locals
              Value: LU_SWEPUB
          Numbering:
            – Type: volume
              Value: 8
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Proceedings of the ACM on Computer Graphics and Interactive Techniques
              Type: main
ResultId 1