Deep learning methods for medical image fusion: A review

The image fusion methods based on deep learning has become a research hotspot in the field of computer vision in recent years. This paper reviews these methods from five aspects: Firstly, the principle and advantages of image fusion methods based on deep learning are expounded; Secondly, the image f...

Full description

Saved in:

Bibliographic Details
Published in:	Computers in biology and medicine Vol. 160; p. 106959
Main Authors:	Zhou, Tao, Cheng, QianRu, Lu, HuiLing, Li, Qi, Zhang, XiangXiang, Qiu, Shi
Format:	Journal Article
Language:	English
Published:	United States Elsevier Ltd 01.06.2023 Elsevier Limited
Subjects:	Algorithms Artificial neural networks Coders Computer vision Convolutional neural network Datasets Deep Learning Encoder-decoder network Encoders-Decoders Feature extraction Generative adversarial network Generative adversarial networks Human error Image Processing, Computer-Assisted - methods Internal Medicine Machine learning Medical image fusion Medical imaging Neural networks Neural Networks, Computer Other Teaching methods Tomography Wavelet transforms Deep learning Medical image fusion Encoder-decoder network Convolutional neural network Generative adversarial network
ISSN:	0010-4825, 1879-0534, 1879-0534
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	The image fusion methods based on deep learning has become a research hotspot in the field of computer vision in recent years. This paper reviews these methods from five aspects: Firstly, the principle and advantages of image fusion methods based on deep learning are expounded; Secondly, the image fusion methods are summarized in two aspects: End-to-End and Non-End-to-End, according to the different tasks of deep learning in the feature processing stage, the non-end-to-end image fusion methods are divided into two categories: deep learning for decision mapping and deep learning for feature extraction. According to the different types of the networks, the end-to-end image fusion methods are divided into three categories: image fusion methods based on Convolutional Neural Network, Generative Adversarial Network, and Encoder-Decoder Network; Thirdly, the application of the image fusion methods based on deep learning in medical image field is summarized from two aspects: method and data set; Fourthly, evaluation metrics commonly used in the field of medical image fusion are sorted out from 14 aspects; Fifthly, the main challenges faced by the medical image fusion are discussed from two aspects: data sets and fusion methods. And the future development direction is prospected. This paper systematically summarizes the image fusion methods based on the deep learning, which has a positive guiding significance for the in-depth study of multi modal medical images. [Display omitted] •Summarizing various deep learning models of medical image fusion.•Discussing the applications of deep learning in medical image fusion field.•Summarizing medical image fusion methods based on deep learning and datasets used in image fusion tasks.•Analyzing the challenges and future development directions of medical image fusion.
Bibliography:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 ObjectType-Review-3 content type line 23
ISSN:	0010-4825 1879-0534 1879-0534
DOI:	10.1016/j.compbiomed.2023.106959