Cascaded transformer U-net for image restoration
•A cascaded multi-stage framework is designed based on the transformer for image restoration.•The self-attention mechanism and cascaded architecture enable the model to achieve long-range dependence modeling and diverse perceptual fields.•A shape perceiving module is designed to improve the model’s...
Gespeichert in:
| Veröffentlicht in: | Signal processing Jg. 206; S. 108902 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Elsevier B.V
01.05.2023
|
| Schlagworte: | |
| ISSN: | 0165-1684 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Zusammenfassung: | •A cascaded multi-stage framework is designed based on the transformer for image restoration.•The self-attention mechanism and cascaded architecture enable the model to achieve long-range dependence modeling and diverse perceptual fields.•A shape perceiving module is designed to improve the model’s adaptabilities for objects with extreme sizes and irregular shapes.•The proposed method achieves superior experimental results on several image restoration tasks, while with less training and inference costs.
Image restoration is one of the most important computer vision tasks, aiming at recovering high-quality images from degraded or low-quality observations. The restoration methods based on convolutional neural networks (CNNs) have achieved attractive performance, however, as convolutions only intake local information, CNN-based methods have limitations in modeling objects in long ranges and extracting global information. In addition, existing one-stage methods damage the performance due to lacking diversified receptive fields. In this paper, we propose a multi-stage cascaded transformer architecture for image restoration. Firstly, the Swin transformer based encoder relying on self-attention is used to improve the modeling ability for long-range objects and outputs hierarchical multi-level semantic features. Then, a shape perceiving module is designed and embedded in the decoder to enhance the representation of irregular objects, Moreover, a multi-stage cascaded encoder-decoder architecture possessing diversified receptive fields is proposed to progressively obtain fine restoration results and thus boost the performance. We conduct extensive experiments, including image deraining, underwater image enhancement, near infrared image colorization and low-light image enhancement. The results show that our proposed method can achieve comparable or better performance than state-of-the-art methods while with less training and inference costs. |
|---|---|
| ISSN: | 0165-1684 |
| DOI: | 10.1016/j.sigpro.2022.108902 |