Sharing Model Framework for Zero‐Shot Sketch‐Based Image Retrieval

Sketch‐based image retrieval (SBIR) is an emerging task in computer vision. Research interests have arisen in solving this problem under the realistic and challenging setting of zero‐shot learning. Given a sketch as a query, the search goal is to retrieve the corresponding photographs in a zero‐shot...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Computer graphics forum Ročník 42; číslo 7
Hlavní autoři: Ho, Yi‐Hsuan, Way, Der‐Lor, Shih, Zen‐Chung
Médium: Journal Article
Jazyk:angličtina
Vydáno: Oxford Blackwell Publishing Ltd 01.10.2023
Témata:
CCS Concepts
Computer vision
Computing methodologies → Machine learning
Demonstration programmes
Image retrieval
Information system → Information retrieval
Semantics
Sketches
ISSN:0167-7055, 1467-8659
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Sketch‐based image retrieval (SBIR) is an emerging task in computer vision. Research interests have arisen in solving this problem under the realistic and challenging setting of zero‐shot learning. Given a sketch as a query, the search goal is to retrieve the corresponding photographs in a zero‐shot scenario. In this paper, we divide the aforementioned challenging work into three tasks and propose a sharing model framework that addresses these problems. First, the weights of the proposed sharing model effectively reduced the modality gap between sketches and photographs. Second, semantic information was used to handle different label spaces during the training and testing stages. The sketch and photograph domains share semantic information. Finally, a memory mechanism is used to reduce the intrinsic variety in sketches, even if they all belong to the same class. Sketches and photographs dominate the embeddings in turn. Because sketches are not limited by language, our ultimate goal is to find a method to replace text searches. We also designed a demonstration program to demonstrate the use of the proposed method in real‐world applications. Our results indicate that the proposed method exhibits considerably higher zero‐shot SBIR performance than do other state‐of‐the‐art methods on the challenging Sketchy, TU‐Berlin, and QuickDraw datasets.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0167-7055
1467-8659
DOI:10.1111/cgf.14947