Explainable analysis of infrared and visible light image fusion based on deep learning

Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimodal image f...

Full description

Saved in:
Bibliographic Details
Published in:Scientific reports Vol. 15; no. 1; pp. 2223 - 10
Main Authors: Yuan, Bo, Sun, Hongyu, Guo, YinJing, Liu, Qiang, Zhan, Xinghao
Format: Journal Article
Language:English
Published: London Nature Publishing Group UK 17.01.2025
Nature Publishing Group
Nature Portfolio
Subjects:
639/166/987
639/705/117
Algorithms
Comparative analysis
Computer vision
Deep learning
Explainability
Humanities and Social Sciences
Image fusion
Image processing
Information processing
Light
Local explanations
Machine learning
multidisciplinary
Neural networks
Science
Science (multidisciplinary)
Explainability
Image fusion
Machine learning
Local explanations
ISSN:2045-2322, 2045-2322
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimodal image fusion model was proposed based on the advantages of convolutional neural networks (CNN) for local context extraction and Transformer global attention mechanism. Secondly, to enhance the explainability of the model, the Delta Debugging Fuse Image (DDFImage) algorithm was employed for generating local explanatory information. Finally, we gain deeper insights into the internal workings of the model through feature importance analysis of the generated explanatory fusion images. Comparative analysis with other explainability algorithms demonstrates the superior performance of our algorithm. This comprehensive approach not only improves the explainability of the model but also provides more reference for practical application of the model.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-024-79684-6