Explainable analysis of infrared and visible light image fusion based on deep learning

Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimodal image f...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Scientific reports Jg. 15; H. 1; S. 2223 - 10
Hauptverfasser: Yuan, Bo, Sun, Hongyu, Guo, YinJing, Liu, Qiang, Zhan, Xinghao
Format: Journal Article
Sprache:Englisch
Veröffentlicht: London Nature Publishing Group UK 17.01.2025
Nature Publishing Group
Nature Portfolio
Schlagworte:
639/166/987
639/705/117
Algorithms
Comparative analysis
Computer vision
Deep learning
Explainability
Humanities and Social Sciences
Image fusion
Image processing
Information processing
Light
Local explanations
Machine learning
multidisciplinary
Neural networks
Science
Science (multidisciplinary)
Explainability
Image fusion
Machine learning
Local explanations
ISSN:2045-2322, 2045-2322
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimodal image fusion model was proposed based on the advantages of convolutional neural networks (CNN) for local context extraction and Transformer global attention mechanism. Secondly, to enhance the explainability of the model, the Delta Debugging Fuse Image (DDFImage) algorithm was employed for generating local explanatory information. Finally, we gain deeper insights into the internal workings of the model through feature importance analysis of the generated explanatory fusion images. Comparative analysis with other explainability algorithms demonstrates the superior performance of our algorithm. This comprehensive approach not only improves the explainability of the model but also provides more reference for practical application of the model.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-024-79684-6