AVAFN-adaptive variational autoencoder fusion network for multispectral image

Image fusion involves extracting and combining the most meaningful information from images captured by different sensors, aiming to create a single image beneficial for further applications. The development of deep learning has significantly advanced image fusion, with neural networks' powerful...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications Vol. 83; no. 41; pp. 89297 - 89315
Main Authors: Chu, Wen-Lin, Tu, Ching-Che, Jian, Bo-Lin
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2024
Springer Nature B.V
Subjects:
1237: Advanced Deep Learning for Computer Vision and Multimedia Applications
Computer Communication Networks
Computer Science
Computer vision
Data Structures and Information Theory
Feature extraction
Image quality
Image reconstruction
Machine learning
Multimedia Information Systems
Neural networks
Special Purpose and Application-Based Systems
Texture
Visual perception
Visual tasks
Variational autoencoder
Multi-input image fusion
Image fusion
Adaptive loss assignment method
ISSN:1573-7721, 1380-7501, 1573-7721
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Image fusion involves extracting and combining the most meaningful information from images captured by different sensors, aiming to create a single image beneficial for further applications. The development of deep learning has significantly advanced image fusion, with neural networks' powerful feature extraction and reconstruction capabilities broadening the prospects for fused images. This study introduces the Adaptive Variational Autoencoder Image Fusion Network (AVAFN), enhancing the Variational Autoencoder (VAE) architecture by incorporating additional convolutional, ReLU, and Dropout layers to improve feature extraction and prevent overfitting. AVAFN introduces a novel loss function that considers global contrast, feature intensity, and structural texture, reducing feature and contrast loss during fusion and preserving detail texture in line with human visual perception. The importance of loss function weights is evident in AVAFN's Adaptive Loss Assignment Method, which adjusts loss weights based on the different feature information of input images, analyzes various depth features, and obtains optimal fusion outcomes. Experimental results reveal that AVAFN excels in multi-modal and multi-exposure image fusion, achieving scene-appropriate fusion results through color channel transformation and effectively extracting depth features in multi-input image fusion tasks through the Concatenation function. These results, confirmed by comparisons with popular fusion methods across three image databases, demonstrate AVAFN's superiority in subjective visual perception and objective fusion metrics. The experiments validate AVAFN's applicability to various image fusion tasks, achieving the desired fusion outcomes even in low-quality images and showcasing its potential in the image fusion domain.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1573-7721
1380-7501
1573-7721
DOI:10.1007/s11042-024-20340-6