Cross-city matters: A multimodal remote sensing benchmark dataset for cross-city semantic segmentation using high-resolution domain adaptation networks

Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e.g., single cities or regions). Yet these AI models tend to meet the performance bottleneck in...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Remote sensing of environment Jg. 299; S. 113856
Hauptverfasser: Hong, Danfeng, Zhang, Bing, Li, Hao, Li, Yuxuan, Yao, Jing, Li, Chenyu, Werner, Martin, Chanussot, Jocelyn, Zipf, Alexander, Zhu, Xiao Xiang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Inc 15.12.2023
Schlagworte:
artificial intelligence
China
class
Cross-city
data collection
Deep learning
Dice loss
domain
Domain adaptation
environment
Germany
High-resolution network
Land cover
Multimodal benchmark datasets
Remote sensing
Segmentation
topology
China
Germany
Deep learning
High-resolution network
Segmentation
Multimodal benchmark datasets
Domain adaptation
Dice loss
Land cover
Remote sensing
Cross-city
ISSN:0034-4257, 1879-0704
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e.g., single cities or regions). Yet these AI models tend to meet the performance bottleneck in the case studies across cities or regions, due to the lack of diverse RS information and cutting-edge solutions with high generalization ability. To this end, we build a new set of multimodal remote sensing benchmark datasets (including hyperspectral, multispectral, SAR) for the study purpose of the cross-city semantic segmentation task (called C2Seg dataset), which consists of two cross-city scenes, i.e., Berlin-Augsburg (in Germany) and Beijing-Wuhan (in China). Beyond the single city, we propose a high-resolution domain adaptation network, HighDAN for short, to promote the AI model's generalization ability from the multi-city environments. HighDAN is capable of retaining the spatially topological structure of the studied urban scene well in a parallel high-to-low resolution fusion fashion but also closing the gap derived from enormous differences of RS image representations between different cities by means of adversarial learning. In addition, the Dice loss is considered in HighDAN to alleviate the class imbalance issue caused by factors across cities. Extensive experiments conducted on the C2Seg dataset show the superiority of our HighDAN in terms of segmentation performance and generalization ability, compared to state-of-the-art competitors. The C2Seg dataset and the semantic segmentation toolbox (involving the proposed HighDAN) will be available publicly at https://github.com/danfenghong/RSE_Cross-city. •A new multimodal remote sensing benchmark for cross-city semantic segmentation.•Propose a high-resolution domain adaptation network for semantic segmentation.•Balance spatial topology, domain gaps, eases city class imbalance with Dice loss.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0034-4257
1879-0704
DOI:10.1016/j.rse.2023.113856