Exploiting Diffusion Prior for Real-World Image Super-Resolution

We present a novel approach to leverage prior knowledge encapsulated in pre-trained text-to-image diffusion models for blind super-resolution. Specifically, by employing our time-aware encoder, we can achieve promising restoration results without altering the pre-trained synthesis model, thereby pre...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision Vol. 132; no. 12; pp. 5929 - 5949
Main Authors: Wang, Jianyi, Yue, Zongsheng, Zhou, Shangchen, Chan, Kelvin C. K., Loy, Chen Change
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2024
Springer
Springer Nature B.V
Subjects:
Artificial Intelligence
Computer Imaging
Computer Science
Computer vision
Controllability
Image Processing and Computer Vision
Image resolution
Methods
Pattern Recognition
Pattern Recognition and Graphics
Vision
Generative prior
Diffusion models
Super-resolution
Image restoration
ISSN:0920-5691, 1573-1405
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a novel approach to leverage prior knowledge encapsulated in pre-trained text-to-image diffusion models for blind super-resolution. Specifically, by employing our time-aware encoder, we can achieve promising restoration results without altering the pre-trained synthesis model, thereby preserving the generative prior and minimizing training cost. To remedy the loss of fidelity caused by the inherent stochasticity of diffusion models, we employ a controllable feature wrapping module that allows users to balance quality and fidelity by simply adjusting a scalar value during the inference process. Moreover, we develop a progressive aggregation sampling strategy to overcome the fixed-size constraints of pre-trained diffusion models, enabling adaptation to resolutions of any size. A comprehensive evaluation of our method using both synthetic and real-world benchmarks demonstrates its superiority over current state-of-the-art approaches. Code and models are available at https://github.com/IceClear/StableSR .
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-024-02168-7