Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

We introduce an approach to image retrieval and auto-tagging that leverages the implicit information about object importance conveyed by the list of keyword tags a person supplies for an image. We propose an unsupervised learning procedure based on Kernel Canonical Correlation Analysis that discover...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International journal of computer vision Ročník 100; číslo 2; s. 134 - 153
Hlavní autoři: Hwang, Sung Ju, Grauman, Kristen
Médium: Journal Article
Jazyk:angličtina
Vydáno: Boston Springer US 01.11.2012
Springer
Springer Nature B.V
Témata:
Analysis
Artificial Intelligence
Computer Imaging
Computer Science
Correlation analysis
Experiments
Human subjects
Image Processing and Computer Vision
Image retrieval
Information management
Information retrieval
International
Keywords
Learning
Learning strategies
Pattern Recognition
Pattern Recognition and Graphics
Perceptions
Queries
Semantics
Studies
Vision
Vision systems
United Kingdom > UK
Image search
Cross-modal retrieval
Image tags
Image retrieval
Multi-modal retrieval
Kernelized canonical correlation analysis
Object recognition
Auto annotation
ISSN:0920-5691, 1573-1405
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We introduce an approach to image retrieval and auto-tagging that leverages the implicit information about object importance conveyed by the list of keyword tags a person supplies for an image. We propose an unsupervised learning procedure based on Kernel Canonical Correlation Analysis that discovers the relationship between how humans tag images (e.g., the order in which words are mentioned) and the relative importance of objects and their layout in the scene. Using this discovered connection, we show how to boost accuracy for novel queries, such that the search results better preserve the aspects a human may find most worth mentioning. We evaluate our approach on three datasets using either keyword tags or natural language descriptions, and quantify results with both ground truth parameters as well as direct tests with human subjects. Our results show clear improvements over approaches that either rely on image features alone, or that use words and image features but ignore the implied importance cues. Overall, our work provides a novel way to incorporate high-level human perception of scenes into visual representations for enhanced image search.
Bibliografie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-011-0494-3