Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval
Sketching enables many exciting applications, notably, image retrieval. The fear-to-sketch problem (i.e., "I can't sketch") has however proven to be fatal for its widespread adoption. This paper tackles this "fear" head on, and for the first time, proposes an auxiliary modul...
Uložené v:
| Vydané v: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) s. 989 - 998 |
|---|---|
| Hlavní autori: | , , , , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
01.01.2022
|
| Predmet: | |
| ISSN: | 1063-6919 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Sketching enables many exciting applications, notably, image retrieval. The fear-to-sketch problem (i.e., "I can't sketch") has however proven to be fatal for its widespread adoption. This paper tackles this "fear" head on, and for the first time, proposes an auxiliary module for existing retrieval models that predominantly lets the users sketch without having to worry. We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch". We consequently design a stroke subset selector that detects noisy strokes, leaving only those which make a positive contribution towards successful retrieval. Our Reinforcement Learning based formulation quantifies the importance of each stroke present in a given subset, based on the extent to which that stroke contributes to retrieval. When combined with pre-trained retrieval models as a pre-processing module, we achieve a significant gain of 8%-10% over standard baselines and in turn report new state-of-the-art performance. Last but not least, we demonstrate the selector once trained, can also be used in a plug-and-play manner to empower various sketch applications in ways that were not previously possible. |
|---|---|
| AbstractList | Sketching enables many exciting applications, notably, image retrieval. The fear-to-sketch problem (i.e., "I can't sketch") has however proven to be fatal for its widespread adoption. This paper tackles this "fear" head on, and for the first time, proposes an auxiliary module for existing retrieval models that predominantly lets the users sketch without having to worry. We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch". We consequently design a stroke subset selector that detects noisy strokes, leaving only those which make a positive contribution towards successful retrieval. Our Reinforcement Learning based formulation quantifies the importance of each stroke present in a given subset, based on the extent to which that stroke contributes to retrieval. When combined with pre-trained retrieval models as a pre-processing module, we achieve a significant gain of 8%-10% over standard baselines and in turn report new state-of-the-art performance. Last but not least, we demonstrate the selector once trained, can also be used in a plug-and-play manner to empower various sketch applications in ways that were not previously possible. |
| Author | Chowdhury, Pinaki Nath Khilji, Abdullah Faiz Ur Rahman Song, Yi-Zhe Sain, Aneeshan Bhunia, Ayan Kumar Koley, Subhadeep Xiang, Tao |
| Author_xml | – sequence: 1 givenname: Ayan Kumar surname: Bhunia fullname: Bhunia, Ayan Kumar email: a.bhunia@surrey.ac.uk organization: University of Surrey,SketchX, CVSSP,United Kingdom – sequence: 2 givenname: Subhadeep surname: Koley fullname: Koley, Subhadeep email: s.koley@surrey.ac.uk organization: University of Surrey,SketchX, CVSSP,United Kingdom – sequence: 3 givenname: Abdullah Faiz Ur Rahman surname: Khilji fullname: Khilji, Abdullah Faiz Ur Rahman organization: iFlyTek-Surrey Joint Research Centre on Artificial Intelligence – sequence: 4 givenname: Aneeshan surname: Sain fullname: Sain, Aneeshan email: a.sain@surrey.ac.uk organization: University of Surrey,SketchX, CVSSP,United Kingdom – sequence: 5 givenname: Pinaki Nath surname: Chowdhury fullname: Chowdhury, Pinaki Nath email: p.chowdhury@surrey.ac.uk organization: University of Surrey,SketchX, CVSSP,United Kingdom – sequence: 6 givenname: Tao surname: Xiang fullname: Xiang, Tao email: t.xiang@surrey.ac.uk organization: University of Surrey,SketchX, CVSSP,United Kingdom – sequence: 7 givenname: Yi-Zhe surname: Song fullname: Song, Yi-Zhe email: y.song@surrey.ac.uk organization: University of Surrey,SketchX, CVSSP,United Kingdom |
| BookMark | eNotjsFKw0AURUdRsK39Al3kBya-N5NMMu601FooKjXosrxMX9rRNJFkVPr3FurqwuFwuENx1rQNC3GNECOCvZm8vSxTZfI8VqBUDICQnYghGpMmxiZGn4oBgtHSWLQXYtz3HwCgFaKx-UDMXj85uK1vNtGvD9v2O0TvbdftD-A2emp9z7Joa-6oCdFRlffU8zqa72jD0ZJD5_mH6ktxXlHd8_h_R6J4mBaTR7l4ns0ndwvplbFBZunhpUPMoSRd6TQlp0uoMlMROCqTjCCrFFqrHZYJgWFnNMG6JOIkZT0SV8esZ-bVV-d31O1XNs8BrdZ_oEFOlg |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52688.2022.00107 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 1665469463 9781665469463 |
| EISSN | 1063-6919 |
| EndPage | 998 |
| ExternalDocumentID | 9880193 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i269t-75202c1180ba3f355ac3b0f76fa0cab47a07f21993c1b4a06ec63a0dbaae45e3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 33 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000867754201024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:15:11 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i269t-75202c1180ba3f355ac3b0f76fa0cab47a07f21993c1b4a06ec63a0dbaae45e3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_9880193 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-01-01 |
| PublicationDateYYYYMMDD | 2022-01-01 |
| PublicationDate_xml | – month: 01 year: 2022 text: 2022-01-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2022 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.4406521 |
| Snippet | Sketching enables many exciting applications, notably, image retrieval. The fear-to-sketch problem (i.e., "I can't sketch") has however proven to be fatal for... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 989 |
| SubjectTerms | Boosting categorization Computational modeling Computer vision grouping and shape analysis; Vision applications and systems Head Image retrieval Pattern recognition Recognition: detection Reinforcement learning retrieval; Segmentation |
| Title | Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval |
| URI | https://ieeexplore.ieee.org/document/9880193 |
| WOSCitedRecordID | wos000867754201024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB7a4sFT1VZ8k4NH12Z3s0nXo8WqIKXUUnsrkxcUbVe2W3-_ye5SEbx4CyEhZCaTmUzmmwG41iKi3IY6oDJN3AMldDLHUASWRVYZTTktUWmzFzEa9efzdNyAmx0WxhhTBp-ZW98s__J1prbeVdZL3WFzBkcTmkLwCqu186fE7iXD036Njgtp2hvMxhOfzMQHcEU-LWdIf9dQKVXIsP2_xQ-g-4PFI-OdljmEhlkfQbs2HkktmpsOPL6-ew64McT7VrNtQd6yPPcopjsyypYbE0yzD-NUU0GqocG902CaPK_clUImZWUtd-y6MB0-TAdPQV0lIVhGPC0CkbjtKZ_JTWJsnfmAKpbUCm6RKpRMIBU28nF6KpQMKTeKx0i1RDQsMfExtNbZ2pwAcdKPiAmikoy5iamMlPZ1MUItpRb0FDqeLIvPKg_GoqbI2d_d57Dv6V65Ky6gVeRbcwl76qtYbvKrknnfDW2bmQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dT8IwEL8gmugTKhi_7YOPTrqtW5mPEhEiLgQX5I1cP5YQlZkx_Pttx4Ix8cW3pmnTtHfXu17vdwdwrbhHw9RVDhVRYB4orpE5htxJmZdKrWhIS1TaZMjjuDOdRqMa3GywMFrrMvhM39pm-ZevMrmyrrJ2ZJjNGBxbsB0w5tE1WmvjUfHNWyaMOhU-zqVRuzsZjW06ExvC5dnEnC79XUWlVCK9xv-W34fWDxqPjDZ65gBqenEIjcp8JJVwLpvw-PJmaWDGEOtdzVYFec3y3OKY7kiczZfaSbJ3bZRTQdZDnXujwxQZfJhLhYzL2lqG8VqQ9B6Sbt-p6iQ4cy-MCocHZnvS5nIT6KfGgEDpC5ryMEUqUTCOlKeejdSTrmBIQy1DH6kSiJoF2j-C-iJb6GMgRv4RMUCUgjEzMRKeVLYyhquEUJyeQNMey-xznQljVp3I6d_dV7DbT56Hs-EgfjqDPUuDtfPiHOpFvtIXsCO_ivkyvywJ-Q1UeJ7g |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Sketching+without+Worrying%3A+Noise-Tolerant+Sketch-Based+Image+Retrieval&rft.au=Bhunia%2C+Ayan+Kumar&rft.au=Koley%2C+Subhadeep&rft.au=Khilji%2C+Abdullah+Faiz+Ur+Rahman&rft.au=Sain%2C+Aneeshan&rft.date=2022-01-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=989&rft.epage=998&rft_id=info:doi/10.1109%2FCVPR52688.2022.00107&rft.externalDocID=9880193 |