Hybrid Sparse Transformer and Feature Alignment for Efficient Image Completion

In this paper, we propose an efficient single‐stage hybrid architecture for image completion. Existing transformer‐based image completion methods often struggle with accurate content restoration, largely due to their ineffective modeling of corrupted channel information and the attention noise intro...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Computer graphics forum Jg. 44; H. 7
Hauptverfasser: Chen, L., Sun, H.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Oxford Blackwell Publishing Ltd 01.10.2025
Schlagworte:
Alignment
CCS Concepts
Computer vision tasks
Computing methodologies → Image processing
Decoupling
Image Completion
Inference
Machine learning
Modules
Neural networks
Texture
ISSN:0167-7055, 1467-8659
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, we propose an efficient single‐stage hybrid architecture for image completion. Existing transformer‐based image completion methods often struggle with accurate content restoration, largely due to their ineffective modeling of corrupted channel information and the attention noise introduced by softmax‐based mechanisms, which results in blurry textures and distorted structures. Additionally, these methods frequently fail to maintain texture consistency, either relying on imprecise mask sampling or incurring substantial computational costs from complex similarity calculations. To address these limitations, we present two key contributions: a Hybrid Sparse Self‐Attention (HSA) module and a Feature Alignment Module (FAM). The HSA module enhances structural recovery by decoupling spatial and channel attention with sparse activation, while the FAM enforces texture consistency by aligning encoder and decoder features via a mask‐free, energy‐gated mechanism without additional inference cost. Our method achieves state‐of‐the‐art image completion results with the fastest inference speed among single‐stage networks, as measured by PSNR, SSIM, FID, and LPIPS on CelebA‐HQ, Places2, and Paris datasets.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0167-7055
1467-8659
DOI:10.1111/cgf.70255