Hybrid Sparse Transformer and Feature Alignment for Efficient Image Completion

In this paper, we propose an efficient single‐stage hybrid architecture for image completion. Existing transformer‐based image completion methods often struggle with accurate content restoration, largely due to their ineffective modeling of corrupted channel information and the attention noise intro...

Full description

Saved in:
Bibliographic Details
Published in:Computer graphics forum Vol. 44; no. 7
Main Authors: Chen, L., Sun, H.
Format: Journal Article
Language:English
Published: Oxford Blackwell Publishing Ltd 01.10.2025
Subjects:
Alignment
CCS Concepts
Computer vision tasks
Computing methodologies → Image processing
Decoupling
Image Completion
Inference
Machine learning
Modules
Neural networks
Texture
ISSN:0167-7055, 1467-8659
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose an efficient single‐stage hybrid architecture for image completion. Existing transformer‐based image completion methods often struggle with accurate content restoration, largely due to their ineffective modeling of corrupted channel information and the attention noise introduced by softmax‐based mechanisms, which results in blurry textures and distorted structures. Additionally, these methods frequently fail to maintain texture consistency, either relying on imprecise mask sampling or incurring substantial computational costs from complex similarity calculations. To address these limitations, we present two key contributions: a Hybrid Sparse Self‐Attention (HSA) module and a Feature Alignment Module (FAM). The HSA module enhances structural recovery by decoupling spatial and channel attention with sparse activation, while the FAM enforces texture consistency by aligning encoder and decoder features via a mask‐free, energy‐gated mechanism without additional inference cost. Our method achieves state‐of‐the‐art image completion results with the fastest inference speed among single‐stage networks, as measured by PSNR, SSIM, FID, and LPIPS on CelebA‐HQ, Places2, and Paris datasets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0167-7055
1467-8659
DOI:10.1111/cgf.70255