Single-Image Super-Resolution for Remote Sensing Images Using a Deep Generative Adversarial Network With Local and Global Attention Mechanisms

Super-resolution (SR) technology is an important way to improve spatial resolution under the condition of sensor hardware limitations. With the development of deep learning (DL), some DL-based SR models have achieved state-of-the-art performance, especially the convolutional neural network (CNN). Ho...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing Vol. 60; pp. 1 - 24
Main Authors: Li, Yadong, Mavromatis, Sebastien, Zhang, Feng, Du, Zhenhong, Sequeira, Jean, Wang, Zhongyi, Zhao, Xianwei, Liu, Renyi
Format: Journal Article
Language:English
Published: New York IEEE 01.01.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Institute of Electrical and Electronics Engineers
Subjects:
Ablation
Artificial neural networks
Biological system modeling
Computational Geometry
Computer Science
Computer Vision and Pattern Recognition
Convolutional neural networks (CNNs)
Deep learning
Dimensions
Earth surface
Feature extraction
generative adversarial network (GAN)
Generative adversarial networks
Graphics
Image Processing
Image reconstruction
Image resolution
local and global attention module
Neural networks
Object recognition
Receptive field
Remote sensing
Resolution
Signal processing algorithms
single-image super super-resolution (SISR)
Spatial discrimination
Spatial resolution
ISSN:0196-2892, 1558-0644
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Super-resolution (SR) technology is an important way to improve spatial resolution under the condition of sensor hardware limitations. With the development of deep learning (DL), some DL-based SR models have achieved state-of-the-art performance, especially the convolutional neural network (CNN). However, considering that remote sensing images usually contain a variety of ground scenes and objects with different scales, orientations, and spectral characteristics, previous works usually treat important and unnecessary features equally or only apply different weights in the local receptive field, which ignores long-range dependencies; it is still a challenging task to exploit features on different levels and reconstruct images with realistic details. To address these problems, an attention-based generative adversarial network (SRAGAN) is proposed in this article, which applies both local and global attention mechanisms. Specifically, we apply local attention in the SR model to focus on structural components of the earth's surface that require more attention, and global attention is used to capture long-range interdependencies in the channel and spatial dimensions to further refine details. To optimize the adversarial learning process, we also use local and global attentions in the discriminator model to enhance the discriminative ability and apply the gradient penalty in the form of hinge loss and loss function that combines <inline-formula> <tex-math notation="LaTeX">L1 </tex-math></inline-formula> pixel loss, <inline-formula> <tex-math notation="LaTeX">L1 </tex-math></inline-formula> perceptual loss, and relativistic adversarial loss to promote rich details. The experiments show that SRAGAN can achieve performance improvements and reconstruct better details compared with current state-of-the-art SR methods. A series of ablation investigations and model analyses validate the efficiency and effectiveness of our method.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2021.3093043