Co-Salient Object Detection with Uncertainty-Aware Group Exchange-Masking
The traditional definition of co-salient object detection (CoSOD) task is to segment the common salient objects in a group of relevant images. Existing CoSOD models by-default adopt the group consensus assumption. This brings about model robustness defect under the condition of irrelevant images in...
Saved in:
| Published in: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 19639 - 19648 |
|---|---|
| Main Authors: | , , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
01.06.2023
|
| Subjects: | |
| ISSN: | 1063-6919 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | The traditional definition of co-salient object detection (CoSOD) task is to segment the common salient objects in a group of relevant images. Existing CoSOD models by-default adopt the group consensus assumption. This brings about model robustness defect under the condition of irrelevant images in the testing image group, which hinders the use of CoSOD models in real-world applications. To address this issue, this paper presents a group exchange-masking (GEM) strategy for robust CoSOD model learning. With two group of image containing different types of salient object as input, the GEM first selects a set of images from each group by the proposed learning based strategy, then these images are exchanged. The proposed feature extraction module considers both the uncertainty caused by the irrelevant images and group consensus in the remaining relevant images. We design a latent variable generator branch which is made of conditional variational autoencoder to generate uncertainly-based global stochastic features. A CoSOD transformer branch is devised to capture the correlation-based local features that contain the group consistency information. At last, the output of two branches are concatenated and fed into a transformer-based decoder, producing robust co-saliency prediction. Extensive evaluations on co-saliency detection with and without irrelevant images demonstrate the superiority of our method over a variety of state-of-the-art methods. |
|---|---|
| AbstractList | The traditional definition of co-salient object detection (CoSOD) task is to segment the common salient objects in a group of relevant images. Existing CoSOD models by-default adopt the group consensus assumption. This brings about model robustness defect under the condition of irrelevant images in the testing image group, which hinders the use of CoSOD models in real-world applications. To address this issue, this paper presents a group exchange-masking (GEM) strategy for robust CoSOD model learning. With two group of image containing different types of salient object as input, the GEM first selects a set of images from each group by the proposed learning based strategy, then these images are exchanged. The proposed feature extraction module considers both the uncertainty caused by the irrelevant images and group consensus in the remaining relevant images. We design a latent variable generator branch which is made of conditional variational autoencoder to generate uncertainly-based global stochastic features. A CoSOD transformer branch is devised to capture the correlation-based local features that contain the group consistency information. At last, the output of two branches are concatenated and fed into a transformer-based decoder, producing robust co-saliency prediction. Extensive evaluations on co-saliency detection with and without irrelevant images demonstrate the superiority of our method over a variety of state-of-the-art methods. |
| Author | Zhang, Kaihua Wu, Yang Liu, Bo Song, Huihui Liu, Dong |
| Author_xml | – sequence: 1 givenname: Yang surname: Wu fullname: Wu, Yang organization: Nanjing University of Information Science and Technology,B-DAT and CICAEET,Nanjing,China – sequence: 2 givenname: Huihui surname: Song fullname: Song, Huihui email: songhuihui@nuist.edu.cn organization: Nanjing University of Information Science and Technology,B-DAT and CICAEET,Nanjing,China – sequence: 3 givenname: Bo surname: Liu fullname: Liu, Bo organization: Walmart Global Tech,Sunnyvale,CA,USA,94086 – sequence: 4 givenname: Kaihua surname: Zhang fullname: Zhang, Kaihua organization: Nanjing University of Information Science and Technology,B-DAT and CICAEET,Nanjing,China – sequence: 5 givenname: Dong surname: Liu fullname: Liu, Dong organization: Netflix Inc,Los Gatos,CA,USA,95032 |
| BookMark | eNotzN1OwjAYANBqNBFxb8DFXqD4td36c0kmIgkGo-It6bqvUMSObDXI20uiV-fu3JKr2EYkZMRgzBiY--rj5bXkipsxBy7GwLRmFyQzymhRggDGjb4kAwZSUGmYuSFZ3-8AQHDGpNEDMq9a-mb3AWPKl_UOXcofMJ0JbcyPIW3zVXTYJRtiOtHJ0XaYz7r2-5BPf9zWxg3SZ9t_hri5I9fe7nvM_h2S1eP0vXqii-VsXk0WNHAoEvVoi9LWWnlX1FBIaRw2NehGOqGtYlZBU3jtGmFKXzquGmaNseiU8txKL4Zk9PcGRFwfuvBlu9OawXnXhRC_tgJQ2Q |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52729.2023.01881 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 9798350301298 |
| EISSN | 1063-6919 |
| EndPage | 19648 |
| ExternalDocumentID | 10204843 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: NSFC grantid: 62276141,61872189 funderid: 10.13039/501100001809 – fundername: National Key Research and Development Program of China grantid: 2018AAA0100400 funderid: 10.13039/501100012166 |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i204t-fea45ab87fc4b04669cedb08d6c38a71a70d4f8cd395f5c27d1a99aec77f2a6f3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 17 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001062531303090&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:56:31 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-fea45ab87fc4b04669cedb08d6c38a71a70d4f8cd395f5c27d1a99aec77f2a6f3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_10204843 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-June |
| PublicationDateYYYYMMDD | 2023-06-01 |
| PublicationDate_xml | – month: 06 year: 2023 text: 2023-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.3812983 |
| Snippet | The traditional definition of co-salient object detection (CoSOD) task is to segment the common salient objects in a group of relevant images. Existing CoSOD... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 19639 |
| SubjectTerms | Generators Object detection Reliability engineering Robustness Self-supervised or unsupervised representation learning Stochastic processes Transformers Uncertainty |
| Title | Co-Salient Object Detection with Uncertainty-Aware Group Exchange-Masking |
| URI | https://ieeexplore.ieee.org/document/10204843 |
| WOSCitedRecordID | wos001062531303090&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwMhECa28eCpPmp8h4NX6j5gGY6mttFEa6PW9NawPJJets126-PfC-xavXjwRggBMmQyMx_zzSB0mScqstQAAWE1oZZ6oIlJApQx5-FTsDwQhe_5aATTqRg3ZPXAhTHGhOQz0_PD8JevF2rtoTKn4b7MLE1bqMV5VpO1NoBK6kKZTEBDj4sjcdV_HT-xxHmPPd8jvBfF4KtX_2qiEmzIsPPP03dR94eNh8cbO7OHtkyxjzqN-4gb5VwdoLv-gjw7t9rtgh9zj6_gG1OFVKsCe7wVT9zCkAFQfZLrd1kaHKAnPPio-b_kQa48dN5Fk-HgpX9Lmk4JZO7uVBFrJGUyB24VzV3EmwlldB6BzlQKkseSR5paUDoVzDKVcB1LIaRRnNtEZjY9RO1iUZgjhK2SCrIcTEwVpSqWIDQDSaVNmeaJPUZdL5rZsi6GMfuWyskf86dox0u_zq46Q-2qXJtztK3eqvmqvAhP-AWa-J3w |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JT0IxEJ4omugJF4y7PXgtvqV9bY8GMRABiYLhRvq6JFzAwMPl39uWJ3rx4K1pmrbpZDIzX-ebAbjOExVZYjjmwmpMLPFAE5WYE0qdh0-4ZYEo3GG9Hh-NRL8kqwcujDEmJJ-Zuh-Gv3w9U0sPlTkN92VmSboJW5SQJFrRtdaQSuqCmUzwkiAXR-Km8dJ_oonzH-u-S3g9irmvX_2rjUqwIvfVf56_B7UfPh7qry3NPmyY6QFUSwcSleq5OIR2Y4afnWPtdkGPuUdY0J0pQrLVFHnEFQ3dwpADUHzi23c5NyiAT6j5sWIA465cePC8BsP75qDRwmWvBDxxdyqwNZJQmXNmFcldzJsJZXQecZ2plEsWSxZpYrnSqaCWqoTpWAohjWLMJjKz6RFUprOpOQZklVQ8y7mJiSJExZILTbkk0qZUs8SeQM0_zfh1VQ5j_P0qp3_MX8FOa9DtjDvt3sMZ7HpJrHKtzqFSzJfmArbVWzFZzC-DOL8AAJShNw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Co-Salient+Object+Detection+with+Uncertainty-Aware+Group+Exchange-Masking&rft.au=Wu%2C+Yang&rft.au=Song%2C+Huihui&rft.au=Liu%2C+Bo&rft.au=Zhang%2C+Kaihua&rft.date=2023-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=19639&rft.epage=19648&rft_id=info:doi/10.1109%2FCVPR52729.2023.01881&rft.externalDocID=10204843 |