Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment

Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. However, existing...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) S. 11194 - 11203
Hauptverfasser: Xiao, Jiayu, Li, Liang, Wang, Chaofei, Zha, Zheng-Jun, Huang, Qingming
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.06.2022
Schlagworte:
Adaptation models
Codes
Computational modeling
Computer vision
Correlation
Image and video synthesis and generation; Transfer/low-shot/long-tail learning
Image coding
Training
ISSN:1063-6919
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. However, existing methods are prone to model overfitting and collapse in extremely few shot setting (less than 10). To solve this problem, we propose a relaxed spatial structural alignment (RSSA) method to calibrate the target generative models during the adaption. We design a cross-domain spatial structural consistency loss comprising the self-correlation and disturbance correlation consistency loss. It helps align the spatial structural information between the synthesis image pairs of the source and target domains. To relax the cross-domain alignment, we compress the original latent space of generative models to a subspace. Image pairs generated from the subspace are pulled closer. Qualitative and quantitative experiments show that our method consistently surpasses the state-of-the-art methods in few shot setting. Our source code: https://github.com/StevenShaw1999/RSSA.
AbstractList Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. However, existing methods are prone to model overfitting and collapse in extremely few shot setting (less than 10). To solve this problem, we propose a relaxed spatial structural alignment (RSSA) method to calibrate the target generative models during the adaption. We design a cross-domain spatial structural consistency loss comprising the self-correlation and disturbance correlation consistency loss. It helps align the spatial structural information between the synthesis image pairs of the source and target domains. To relax the cross-domain alignment, we compress the original latent space of generative models to a subspace. Image pairs generated from the subspace are pulled closer. Qualitative and quantitative experiments show that our method consistently surpasses the state-of-the-art methods in few shot setting. Our source code: https://github.com/StevenShaw1999/RSSA.
Author Huang, Qingming
Zha, Zheng-Jun
Xiao, Jiayu
Li, Liang
Wang, Chaofei
Author_xml – sequence: 1
  givenname: Jiayu
  surname: Xiao
  fullname: Xiao, Jiayu
  email: jiayu.xiao@vipl.ict.ac.cn
  organization: Inst. of Comput. Tech., CAS,Key Lab of Intell. Info. Process.,Beijing,China
– sequence: 2
  givenname: Liang
  surname: Li
  fullname: Li, Liang
  email: liang.li@ict.ac.cn
  organization: Inst. of Comput. Tech., CAS,Key Lab of Intell. Info. Process.,Beijing,China
– sequence: 3
  givenname: Chaofei
  surname: Wang
  fullname: Wang, Chaofei
  email: wangcf18@mails.tsinghua.edu.cn
  organization: Tsinghua Univesity,Department of Automation
– sequence: 4
  givenname: Zheng-Jun
  surname: Zha
  fullname: Zha, Zheng-Jun
  email: zhazj@ustc.edu.cn
  organization: University of Science and Technology of China,China
– sequence: 5
  givenname: Qingming
  surname: Huang
  fullname: Huang, Qingming
  email: qmhuang@ucas.ac.cn
  organization: Inst. of Comput. Tech., CAS,Key Lab of Intell. Info. Process.,Beijing,China
BookMark eNotjF1LwzAYRqMouM39Ar3IH2jNm6T5uBzDTXGirOrtSJO3Guna0WVT_70FvXoOh8MzJmdt1yIh18ByAGZv5m_P64IrY3LOOM_Z4PgJGYNShVRWKnFKRsCUyJQFe0Gm-_0nY0xwAGXNiDws8IuWH12iS2yxdykekT52ARs6C26XYtfSY3R0jY37xkDL3ZC4hpapP_h06AecNfG93WKbLsl57Zo9Tv93Ql4Xty_zu2z1tLyfz1ZZ5EykjFcSofZKg9RM69qAd8xbxSUaV6vCeBmCZo7zYHyoAAsBlUFZgfXaVJWYkKu_34iIm10ft67_2VijrVJC_AIYc1AH
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/CVPR52688.2022.01092
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
EISBN 1665469463
9781665469463
EISSN 1063-6919
EndPage 11203
ExternalDocumentID 9879663
Genre orig-research
GrantInformation_xml – fundername: National Natural Science Foundation of China
  grantid: U21B2038,61931008,61732007
  funderid: 10.13039/501100001809
– fundername: National Key R&D Program of China
  grantid: 2018AAA0102000
  funderid: 10.13039/501100012166
GroupedDBID 6IE
6IH
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IJVOP
OCL
RIE
RIL
RIO
ID FETCH-LOGICAL-i203t-2b4e1fc67147077f81ca0c9624e8af658c4dd70a22d8cdb1e531b8e4b19c78bb3
IEDL.DBID RIE
ISICitedReferencesCount 45
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000870759104028&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:15:09 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-2b4e1fc67147077f81ca0c9624e8af658c4dd70a22d8cdb1e531b8e4b19c78bb3
PageCount 10
ParticipantIDs ieee_primary_9879663
PublicationCentury 2000
PublicationDate 2022-June
PublicationDateYYYYMMDD 2022-06-01
PublicationDate_xml – month: 06
  year: 2022
  text: 2022-June
PublicationDecade 2020
PublicationTitle Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online)
PublicationTitleAbbrev CVPR
PublicationYear 2022
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003211698
Score 2.5199208
Snippet Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a...
SourceID ieee
SourceType Publisher
StartPage 11194
SubjectTerms Adaptation models
Codes
Computational modeling
Computer vision
Correlation
Image and video synthesis and generation; Transfer/low-shot/long-tail learning
Image coding
Training
Title Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment
URI https://ieeexplore.ieee.org/document/9879663
WOSCitedRecordID wos000870759104028&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NSwMxEB3a4sFT1Vb8JgePbrvJxk1yLMUiKKVYLb2VbDbRhdIWW6s_30x2qQhevIVACJmQmbzkzRuAaw9MPOqwLtKKu4hboyIpjUNnqJ2PiFwFbs7kUQyHcjpVoxrc7HJhrLWBfGY72Ax_-fnSfOBTWdfjY387T-pQFyItc7V27ymJRzKpklV2HI1Vtz8ZPaGYCRK4GOvgHxD7VUMlhJBB83-TH0D7JxePjHZR5hBqdnEEzerySKqjuW7Bw8B-kvHbckNKJWl0YwQrnc1JL9fBMZBtoQmy375wJFKp9ZyMg4Asim-Q3rx4DeSANrwM7p7791FVKSEqWJxsIpZxS51JBeUiFsJJanRsVMq4ld7kt9LwPBexZgyLFWXU-pOXScszqoyQWZYcQ2OxXNgTILlDBbtUaerxqjZMaSSCSukEi9MsTU6hhbaZrUoxjFlllrO_u89hH41fcqsuoOGXZC9hz2w3xfr9KuzgN0CQnDk
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEJ0gmugJFYzf9uDRhd1u2bZHQiQYkBBBwo10u61uQsAIoj_fzu4GY-LFW9OkaTpNZ_raN28Abh0wcajDWE9JZj1mtPSE0BadobIuIjKZcXMmfT4YiOlUDktwt82FMcZk5DNTx2b2l58s9Qc-lTUcPna383AHdpuMUT_P1tq-qIQOy0RSFPlxgS8b7cnwCeVMkMJFaR1_geivKipZEOlU_jf9IdR-svHIcBtnjqBkFsdQKa6PpDicqyr0OuaTjF6Xa5JrSaMjI1jrbE5aicpcA9mkiiD_7QtHIplazckok5BF-Q3SmqcvGT2gBs-d-3G76xW1EryU-uHaozEzgdURDxj3Obci0MrXMqLMCGf0ptAsSbivKMVyRXFg3NmLhWFxIDUXcRyeQHmxXJhTIIlFDbtIqsAhVqWpVEgFFcJy6kdxFJ5BFW0ze8vlMGaFWc7_7r6B_e74sT_rPwx6F3CAG5EzrS6h7JZnrmBPb9bp6v06281vkTifgA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Few+Shot+Generative+Model+Adaption+via+Relaxed+Spatial+Structural+Alignment&rft.au=Xiao%2C+Jiayu&rft.au=Li%2C+Liang&rft.au=Wang%2C+Chaofei&rft.au=Zha%2C+Zheng-Jun&rft.date=2022-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=11194&rft.epage=11203&rft_id=info:doi/10.1109%2FCVPR52688.2022.01092&rft.externalDocID=9879663