Multi-modality MRI fusion with patch complementary pre-training for internet of medical things-based smart healthcare
Magnetic Resonance Imaging (MRI) is a pivotal neuroimaging technique capable of generating images with various contrasts, known as multi-modal images. The integration of these diverse modalities is essential for improving model performance across various tasks. However, in real clinical scenarios, a...
Gespeichert in:
| Veröffentlicht in: | Information fusion Jg. 107; S. 102342 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Elsevier B.V
01.07.2024
|
| Schlagworte: | |
| ISSN: | 1566-2535, 1872-6305 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Magnetic Resonance Imaging (MRI) is a pivotal neuroimaging technique capable of generating images with various contrasts, known as multi-modal images. The integration of these diverse modalities is essential for improving model performance across various tasks. However, in real clinical scenarios, acquiring MR images for all modalities is frequently hindered by factors such as patient comfort and scanning costs. Therefore, effectively fusing different modalities to synthesize missing modalities has become a research hot-spot in the field of smart healthcare, particularly in the context of the Internet of Medical Things (IoMT). In this study, we introduce a multi-modal coordinated fusion network (MCF-Net) with Patch Complementarity Pre-training. This network leverages the complementarity and correlation between different modalities to make the fusion of multi-modal MR images, addressing challenges in the IoMT. Specifically, we first employ a Patch Complementarity Mask Autoencoder (PC-MAE) for self-supervised pre-training. The complementarity learning mechanism is introduced to align masks and visual annotations between two modalities. Subsequently, a dual-branch MAE architecture and a shared encoder–decoder are adopted to facilitate cross-modal interactions within mask tokens. Furthermore, during the fine-tuning phase, we incorporate an Attention-Driven Fusion (ADF) module into the MCF-Net. This module synthesizes missing modal images by fusion of multi-modal features from the pre-trained PC-MAE encoder. Additionally, we leverage the pre-trained encoder to extract high-level features from both synthetic and corresponding real images, ensuring consistency throughout the training process. Our experimental findings showcase a notable enhancement in performance across various modalities with our fusion method, outperforming state-of-the-art techniques.
•Propose a multi-modal medical image fusion framework with patch complementary pre-training•Design a novel masking alignment strategy to learn complementary information between modalities.•Introduce an attention-driven fusion module to aggregate multi-modal features. |
|---|---|
| AbstractList | Magnetic Resonance Imaging (MRI) is a pivotal neuroimaging technique capable of generating images with various contrasts, known as multi-modal images. The integration of these diverse modalities is essential for improving model performance across various tasks. However, in real clinical scenarios, acquiring MR images for all modalities is frequently hindered by factors such as patient comfort and scanning costs. Therefore, effectively fusing different modalities to synthesize missing modalities has become a research hot-spot in the field of smart healthcare, particularly in the context of the Internet of Medical Things (IoMT). In this study, we introduce a multi-modal coordinated fusion network (MCF-Net) with Patch Complementarity Pre-training. This network leverages the complementarity and correlation between different modalities to make the fusion of multi-modal MR images, addressing challenges in the IoMT. Specifically, we first employ a Patch Complementarity Mask Autoencoder (PC-MAE) for self-supervised pre-training. The complementarity learning mechanism is introduced to align masks and visual annotations between two modalities. Subsequently, a dual-branch MAE architecture and a shared encoder–decoder are adopted to facilitate cross-modal interactions within mask tokens. Furthermore, during the fine-tuning phase, we incorporate an Attention-Driven Fusion (ADF) module into the MCF-Net. This module synthesizes missing modal images by fusion of multi-modal features from the pre-trained PC-MAE encoder. Additionally, we leverage the pre-trained encoder to extract high-level features from both synthetic and corresponding real images, ensuring consistency throughout the training process. Our experimental findings showcase a notable enhancement in performance across various modalities with our fusion method, outperforming state-of-the-art techniques.
•Propose a multi-modal medical image fusion framework with patch complementary pre-training•Design a novel masking alignment strategy to learn complementary information between modalities.•Introduce an attention-driven fusion module to aggregate multi-modal features. |
| ArticleNumber | 102342 |
| Author | AlQahtani, Salman A. Lyu, Jun Hossain, M. Shamim Chen, Xiudong |
| Author_xml | – sequence: 1 givenname: Jun orcidid: 0000-0003-1989-1360 surname: Lyu fullname: Lyu, Jun email: ljdream0710@pku.edu.cn organization: School of Computer and Control Engineering, Yantai University, Yantai, 264005, China – sequence: 2 givenname: Xiudong surname: Chen fullname: Chen, Xiudong email: chenxiudong@s.ytu.edu.cn organization: School of Computer and Control Engineering, Yantai University, Yantai, 264005, China – sequence: 3 givenname: Salman A. surname: AlQahtani fullname: AlQahtani, Salman A. email: salmanq@ksu.edu.sa, alqahtani.s@gmail.com organization: Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia – sequence: 4 givenname: M. Shamim surname: Hossain fullname: Hossain, M. Shamim email: mshossain@ksu.edu.sa organization: Department of Software Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, 12372, Saudi Arabia |
| BookMark | eNqFkM1KAzEURoNUsK2-gYu8wNTcpJOZuhCk-FNoEUTXIc3ccVJmMiVJlb69KePKha5ySe75yHcmZOR6h4RcA5sBA3mzm1lX14cw44zP0xUXc35GxlAWPJOC5aM051JmPBf5BZmEsGMMCiZgTA6bQxtt1vWVbm080s3riqYk2zv6ZWND9zqahpq-27fYoYvaH-neYxa9ts66D1r3nloX0TuMtK9ph5U1uqWxSa8h2-qAFQ2d9pE2qNvYGO3xkpzXug149XNOyfvjw9vyOVu_PK2W9-vMCCZjghnP62JR6qIUBSLj22oLxoithIpDVVYwX4CBHEHneSEBUACrteQyrfBSTMntkGt8H4LHWhkbdUztTv9vFTB1Eqh2ahCoTgLVIDDB81_w3ttU5PgfdjdgmIp9WvQqGIvOJC8eTVRVb_8O-AYGRJDS |
| CitedBy_id | crossref_primary_10_3390_math13111768 crossref_primary_10_1016_j_bspc_2024_106857 crossref_primary_10_1016_j_inffus_2025_103302 crossref_primary_10_1109_TMI_2025_3550206 crossref_primary_10_1016_j_bspc_2025_108626 crossref_primary_10_1109_TIM_2025_3545534 |
| Cites_doi | 10.1109/TMI.2022.3167808 10.1109/TMI.2020.3046444 10.1109/JBHI.2021.3082541 10.3389/fonc.2022.942511 10.1002/hbm.24428 10.1109/TMI.2020.2987026 10.1109/TMI.2019.2901750 10.1016/j.inffus.2021.06.007 10.1016/j.inffus.2022.03.010 10.1109/TMI.2018.2884053 10.1007/s00530-017-0561-x 10.1016/j.inffus.2022.12.027 10.1016/j.infrared.2016.01.009 10.1016/j.compbiomed.2023.106738 10.1016/j.inffus.2021.06.008 10.1162/0899766042321814 10.1109/TPAMI.2010.183 10.1109/TMI.2020.2975344 10.1109/TMI.2019.2895894 10.1109/TMI.2014.2377694 10.1016/j.inffus.2021.02.019 10.1117/1.JMI.6.1.014005 10.1109/JBHI.2019.2922986 |
| ContentType | Journal Article |
| Copyright | 2024 Elsevier B.V. |
| Copyright_xml | – notice: 2024 Elsevier B.V. |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.inffus.2024.102342 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Mathematics |
| EISSN | 1872-6305 |
| ExternalDocumentID | 10_1016_j_inffus_2024_102342 S1566253524001209 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 29I 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABFNM ABJNI ABMAC ABXDB ABYKQ ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV AKRWK ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EJD EO8 EO9 EP2 EP3 F0J F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HVGLF HZ~ IHE J1W JJJVA KOM M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SST SSV SSZ T5K UHS ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKYEP ANKPU APXCP CITATION EFKBS EFLBG ~HD |
| ID | FETCH-LOGICAL-c306t-ba025f798a7837ee02bdb1cc3b61d21d8d1491c15e1a557611e310fa6263b6283 |
| ISICitedReferencesCount | 8 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001208236800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1566-2535 |
| IngestDate | Sat Nov 29 07:05:17 EST 2025 Tue Nov 18 22:16:45 EST 2025 Sat Apr 06 16:25:38 EDT 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Magnetic resonance imaging (MRI) Multi-modal medical image fusion Smart healthcare Internet of medical things (IoMT) |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c306t-ba025f798a7837ee02bdb1cc3b61d21d8d1491c15e1a557611e310fa6263b6283 |
| ORCID | 0000-0003-1989-1360 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_inffus_2024_102342 crossref_primary_10_1016_j_inffus_2024_102342 elsevier_sciencedirect_doi_10_1016_j_inffus_2024_102342 |
| PublicationCentury | 2000 |
| PublicationDate | July 2024 2024-07-00 |
| PublicationDateYYYYMMDD | 2024-07-01 |
| PublicationDate_xml | – month: 07 year: 2024 text: July 2024 |
| PublicationDecade | 2020 |
| PublicationTitle | Information fusion |
| PublicationYear | 2024 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Li, Zhou, He, Zhou, Shen (b33) 2023 Liu, Pasumarthi, Duffy, Gong, Datta, Zaharchuk (b32) 2023 Yan, Wang, Chen, Lyu (b31) 2022; 12 Fu, Wu, Durrani (b8) 2021; 72 Zhang, Xu, Tian, Jiang, Ma (b4) 2021; 76 Hardoon, Szedmak, Shawe-Taylor (b36) 2004; 16 Tao, Velasquez (b17) 2022; 83 Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin (b26) 2017; vol. 30 Huang, Shao, Frangi (b23) 2017 Wang, Zhou, Yu, Wang, Zu, Lalush, Lin, Wu, Zhou, Shen (b25) 2018; 38 Zhou, Liu, Fu, Wang, Shen, Shao, Shen (b1) 2019 Menze, Jakab, Bauer, Kalpathy-Cramer, Farahani, Kirby, Burren, Porz, Slotboom, Wiest (b42) 2014; 34 Wang, Gong, Banerjee, Martin, Tong, Choi, Chen, Wintermark, Pauly, Zaharchuk (b10) 2020; 39 Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232. Dar, Yurt, Karacan, Erdem, Erdem, Cukur (b19) 2019; 38 Chen, He, Frey, Li, Du (b28) 2021 Chen, Lu, Yu, Luo, Adeli, Wang, Lu, Yuille, Zhou (b27) 2021 Yang, Lin, Wang, Li, Cheng (b12) 2019; 24 Dalmaz, Yurt, Çukur (b29) 2022; 41 Li, Xu, Lu, Qi (b35) 2023; 11 Lin, Liu, Fuh (b37) 2010; 33 Wolterink, Dinkla, Savenije, Seevinck, van den Berg, Išgum (b20) 2017 Zhou, Fu, Chen, Shen, Shao (b14) 2020; 39 Umirzakova, Ahmad, Khan, Whangbo (b16) 2023 Zhou, Thung, Zhu, Shen (b38) 2019; 40 Hou, Tang, Zhang, Kong, Zhao (b39) 2019; 32 Li, Xu, Chen, Sun, Bian, Guo, Lu, Qi (b34) 2023; 157 Goodfellow, Pouget-Abadie, Mirza, Xu, Warde-Farley, Ozair, Courville, Bengio (b18) 2014; vol. 27 Yu, Zhou, Wang, Shi, Fripp, Bourgeat (b44) 2019; 38 Peng, Liu, Bin, Shen, Lei (b30) 2021; 26 Sevetlidis, Giuffrida, Tsaftaris (b5) 2016 Zhang, Li, Wang, Wang, Lai, Zhang (b3) 2023; 93 Hossain (b15) 2019; 25 Zhang, He, Guo, Ettehadi, Aw, Semanek, Posner, Laine, Wang (b43) 2021 Shen, Zhu, Wang, Xing, Pauly, Turkbey, Harmon, Sanford, Mehralivand, Choyke (b24) 2020; 40 Wei, Poirion, Bodini, Durrleman, Colliot, Stankoff, Ayache (b41) 2019; 6 Dar, Yurt, Karacan, Erdem, Erdem, Cukur (b7) 2019; 38 Yu, Zhou, Wang, Fripp, Bourgeat (b40) 2018 Wen Wei, Emilie Poirion, Benedetta Bodini, Stanley Durrleman, Olivier Colliot, Bruno Stankoff, Nicholas Ayache, FLAIR MR image synthesis by using 3D fully convolutional networks for multiple sclerosis, in: ISMRM-ESMRMB 2018-Joint Annual Meeting, 2018, pp. 1–6. Bavirisetti, Dhuli (b6) 2016; 76 Olut, Sahin, Demir, Unal (b13) 2018 Muhammad, Alshehri, Karray, El Saddik, Alsulaiman, Falk (b2) 2021; 76 Hossain (10.1016/j.inffus.2024.102342_b15) 2019; 25 Dar (10.1016/j.inffus.2024.102342_b19) 2019; 38 Huang (10.1016/j.inffus.2024.102342_b23) 2017 Zhou (10.1016/j.inffus.2024.102342_b38) 2019; 40 Zhang (10.1016/j.inffus.2024.102342_b3) 2023; 93 Zhou (10.1016/j.inffus.2024.102342_b14) 2020; 39 Vaswani (10.1016/j.inffus.2024.102342_b26) 2017; vol. 30 Fu (10.1016/j.inffus.2024.102342_b8) 2021; 72 Yan (10.1016/j.inffus.2024.102342_b31) 2022; 12 Hou (10.1016/j.inffus.2024.102342_b39) 2019; 32 Yu (10.1016/j.inffus.2024.102342_b40) 2018 Menze (10.1016/j.inffus.2024.102342_b42) 2014; 34 10.1016/j.inffus.2024.102342_b21 Zhang (10.1016/j.inffus.2024.102342_b43) 2021 Lin (10.1016/j.inffus.2024.102342_b37) 2010; 33 Liu (10.1016/j.inffus.2024.102342_b32) 2023 Yu (10.1016/j.inffus.2024.102342_b44) 2019; 38 10.1016/j.inffus.2024.102342_b22 Yang (10.1016/j.inffus.2024.102342_b12) 2019; 24 Sevetlidis (10.1016/j.inffus.2024.102342_b5) 2016 Bavirisetti (10.1016/j.inffus.2024.102342_b6) 2016; 76 Tao (10.1016/j.inffus.2024.102342_b17) 2022; 83 10.1016/j.inffus.2024.102342_b9 Wang (10.1016/j.inffus.2024.102342_b10) 2020; 39 Dalmaz (10.1016/j.inffus.2024.102342_b29) 2022; 41 Dar (10.1016/j.inffus.2024.102342_b7) 2019; 38 Li (10.1016/j.inffus.2024.102342_b35) 2023; 11 Chen (10.1016/j.inffus.2024.102342_b28) 2021 Olut (10.1016/j.inffus.2024.102342_b13) 2018 Umirzakova (10.1016/j.inffus.2024.102342_b16) 2023 Wang (10.1016/j.inffus.2024.102342_b25) 2018; 38 Li (10.1016/j.inffus.2024.102342_b33) 2023 Muhammad (10.1016/j.inffus.2024.102342_b2) 2021; 76 10.1016/j.inffus.2024.102342_b11 Chen (10.1016/j.inffus.2024.102342_b27) 2021 Wolterink (10.1016/j.inffus.2024.102342_b20) 2017 Li (10.1016/j.inffus.2024.102342_b34) 2023; 157 Zhou (10.1016/j.inffus.2024.102342_b1) 2019 Zhang (10.1016/j.inffus.2024.102342_b4) 2021; 76 Goodfellow (10.1016/j.inffus.2024.102342_b18) 2014; vol. 27 Hardoon (10.1016/j.inffus.2024.102342_b36) 2004; 16 Wei (10.1016/j.inffus.2024.102342_b41) 2019; 6 Shen (10.1016/j.inffus.2024.102342_b24) 2020; 40 Peng (10.1016/j.inffus.2024.102342_b30) 2021; 26 |
| References_xml | – volume: vol. 27 year: 2014 ident: b18 article-title: Generative adversarial nets publication-title: Advances in Neural Information Processing Systems – volume: 40 start-page: 1001 year: 2019 end-page: 1016 ident: b38 article-title: Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis publication-title: Hum. Brain Mapp. – volume: 76 start-page: 355 year: 2021 end-page: 375 ident: b2 article-title: A comprehensive survey on multimodal medical signals fusion for smart healthcare systems publication-title: Inf. Fusion – year: 2023 ident: b33 article-title: Multi-scale transformer network with edge-aware pre-training for cross-modality MR image synthesis publication-title: IEEE Trans. Med. Imaging – start-page: 147 year: 2018 end-page: 154 ident: b13 article-title: Generative adversarial training for MRA image synthesis using multi-contrast MRI publication-title: PRedictive Intelligence in MEdicine: First International Workshop, PRIME 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 1 – volume: 38 start-page: 1750 year: 2019 end-page: 1762 ident: b44 article-title: Ea-GANs: Edge-aware generative adversarial networks for cross-modality MR image synthesis publication-title: IEEE Trans. Med. Imaging – volume: 25 start-page: 565 year: 2019 end-page: 575 ident: b15 article-title: Smart healthcare monitoring: A voice pathology detection paradigm for smart cities publication-title: Multimedia Syst. – volume: 83 start-page: 93 year: 2022 end-page: 95 ident: b17 article-title: Multi-source information fusion for smart health with artificial intelligence publication-title: Inf. Fusion – volume: vol. 30 year: 2017 ident: b26 article-title: Attention is all you need publication-title: Advances in Neural Information Processing Systems – year: 2021 ident: b28 article-title: Vit-v-net: Vision transformer for unsupervised volumetric medical image registration – volume: 34 start-page: 1993 year: 2014 end-page: 2024 ident: b42 article-title: The multimodal brain tumor image segmentation benchmark (BRATS) publication-title: IEEE Trans. Med. Imaging – volume: 26 start-page: 27 year: 2021 end-page: 35 ident: b30 article-title: Multi-modality mr image synthesis via confidence-guided aggregation and cross-modality refinement publication-title: IEEE J. Biomed. Health Inf. – year: 2021 ident: b43 article-title: PTNet: A high-resolution infant MRI synthesizer based on transformer – volume: 76 start-page: 323 year: 2021 end-page: 336 ident: b4 article-title: Image fusion meets deep learning: A survey and perspective publication-title: Inf. Fusion – reference: Wen Wei, Emilie Poirion, Benedetta Bodini, Stanley Durrleman, Olivier Colliot, Bruno Stankoff, Nicholas Ayache, FLAIR MR image synthesis by using 3D fully convolutional networks for multiple sclerosis, in: ISMRM-ESMRMB 2018-Joint Annual Meeting, 2018, pp. 1–6. – year: 2021 ident: b27 article-title: Transunet: Transformers make strong encoders for medical image segmentation – volume: 32 year: 2019 ident: b39 article-title: Deep multimodal multilinear fusion with high-order polynomial pooling publication-title: Adv. Neural Inf. Process. Syst. – start-page: 629 year: 2019 end-page: 638 ident: b1 article-title: Deep multi-modal latent representation learning for automated dementia diagnosis publication-title: International Conference on Medical Image Computing and Computer-Assisted Intervention – volume: 40 start-page: 1113 year: 2020 end-page: 1122 ident: b24 article-title: Multi-domain image completion for random missing input data publication-title: IEEE Trans. Med. Imaging – volume: 16 start-page: 2639 year: 2004 end-page: 2664 ident: b36 article-title: Canonical correlation analysis: An overview with application to learning methods publication-title: Neural Comput. – volume: 12 year: 2022 ident: b31 article-title: Swin transformer-based GAN for multi-modal medical image translation publication-title: Front. Oncol. – volume: 6 start-page: 014005 year: 2019 ident: b41 article-title: Fluid-attenuated inversion recovery MRI synthesis from multisequence MRI using three-dimensional fully convolutional networks for multiple sclerosis publication-title: J. Med. Imaging – start-page: 626 year: 2018 end-page: 630 ident: b40 article-title: 3D cGAN based cross-modality MR image synthesis for brain tumor segmentation publication-title: 2018 IEEE 15th International Symposium on Biomedical Imaging – reference: Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232. – volume: 33 start-page: 1147 year: 2010 end-page: 1160 ident: b37 article-title: Multiple kernel learning for dimensionality reduction publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – start-page: 127 year: 2016 end-page: 137 ident: b5 article-title: Whole image synthesis using a deep encoder-decoder network publication-title: Simulation and Synthesis in Medical Imaging: First International Workshop, SASHIMI 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 21, 2016, Proceedings 1 – volume: 72 start-page: 110 year: 2021 end-page: 125 ident: b8 article-title: Image fusion based on generative adversarial network consistent with perception publication-title: Inf. Fusion – volume: 157 year: 2023 ident: b34 article-title: CT synthesis from multi-sequence MRI using adaptive fusion network publication-title: Comput. Biol. Med. – volume: 38 start-page: 2375 year: 2019 end-page: 2388 ident: b7 article-title: Image synthesis in multi-contrast MRI with conditional generative adversarial networks publication-title: IEEE Trans. Med. Imaging – start-page: 14 year: 2017 end-page: 23 ident: b20 article-title: Deep MR to CT synthesis using unpaired data publication-title: Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, QuÉBec City, QC, Canada, September 10, 2017, Proceedings 2 – volume: 11 year: 2023 ident: b35 article-title: CT synthesis from MRI with an improved multi-scale learning network publication-title: Front. Phys. – volume: 39 start-page: 3089 year: 2020 end-page: 3099 ident: b10 article-title: Synthesize high-quality multi-contrast magnetic resonance imaging from multi-echo acquisition using multi-task deep generative model publication-title: IEEE Trans. Med. Imaging – volume: 39 start-page: 2772 year: 2020 end-page: 2781 ident: b14 article-title: Hi-net: Hybrid-fusion network for multi-modal MR image synthesis publication-title: IEEE Trans. Med. Imaging – year: 2023 ident: b16 article-title: Medical image super-resolution for smart healthcare applications: A comprehensive survey publication-title: Inf. Fusion – volume: 41 start-page: 2598 year: 2022 end-page: 2614 ident: b29 article-title: ResViT: Residual vision transformers for multimodal medical image synthesis publication-title: IEEE Trans. Med. Imaging – volume: 38 start-page: 1328 year: 2018 end-page: 1339 ident: b25 article-title: 3D auto-context-based locality adaptive multi-modality GANs for PET synthesis publication-title: IEEE Trans. Med. Imaging – volume: 24 start-page: 855 year: 2019 end-page: 865 ident: b12 article-title: Bi-modality medical image synthesis using semi-supervised sequential generative adversarial networks publication-title: IEEE J. Biomed. Health Inform. – start-page: 89 year: 2017 end-page: 98 ident: b23 article-title: DOTE: Dual convolutional filter learning for super-resolution and cross-modality synthesis in MRI publication-title: Medical Image Computing and Computer Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III 20 – year: 2023 ident: b32 article-title: One model to synthesize them all: Multi-contrast multi-scale transformer for missing data imputation publication-title: IEEE Trans. Med. Imaging – volume: 93 start-page: 192 year: 2023 end-page: 208 ident: b3 article-title: A multi-source information fusion model for outlier detection publication-title: Inf. Fusion – volume: 76 start-page: 52 year: 2016 end-page: 64 ident: b6 article-title: Two-scale image fusion of visible and infrared images using saliency detection publication-title: Infrared Phys. Technol. – volume: 38 start-page: 2375 year: 2019 end-page: 2388 ident: b19 article-title: Image synthesis in multi-contrast MRI with conditional generative adversarial networks publication-title: IEEE Trans. Med. Imaging – volume: 11 year: 2023 ident: 10.1016/j.inffus.2024.102342_b35 article-title: CT synthesis from MRI with an improved multi-scale learning network publication-title: Front. Phys. – volume: vol. 27 year: 2014 ident: 10.1016/j.inffus.2024.102342_b18 article-title: Generative adversarial nets – volume: 41 start-page: 2598 issue: 10 year: 2022 ident: 10.1016/j.inffus.2024.102342_b29 article-title: ResViT: Residual vision transformers for multimodal medical image synthesis publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2022.3167808 – year: 2021 ident: 10.1016/j.inffus.2024.102342_b27 – year: 2023 ident: 10.1016/j.inffus.2024.102342_b16 article-title: Medical image super-resolution for smart healthcare applications: A comprehensive survey publication-title: Inf. Fusion – start-page: 626 year: 2018 ident: 10.1016/j.inffus.2024.102342_b40 article-title: 3D cGAN based cross-modality MR image synthesis for brain tumor segmentation – volume: 40 start-page: 1113 issue: 4 year: 2020 ident: 10.1016/j.inffus.2024.102342_b24 article-title: Multi-domain image completion for random missing input data publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2020.3046444 – year: 2023 ident: 10.1016/j.inffus.2024.102342_b32 article-title: One model to synthesize them all: Multi-contrast multi-scale transformer for missing data imputation publication-title: IEEE Trans. Med. Imaging – volume: 26 start-page: 27 issue: 1 year: 2021 ident: 10.1016/j.inffus.2024.102342_b30 article-title: Multi-modality mr image synthesis via confidence-guided aggregation and cross-modality refinement publication-title: IEEE J. Biomed. Health Inf. doi: 10.1109/JBHI.2021.3082541 – volume: 12 year: 2022 ident: 10.1016/j.inffus.2024.102342_b31 article-title: Swin transformer-based GAN for multi-modal medical image translation publication-title: Front. Oncol. doi: 10.3389/fonc.2022.942511 – volume: 40 start-page: 1001 issue: 3 year: 2019 ident: 10.1016/j.inffus.2024.102342_b38 article-title: Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis publication-title: Hum. Brain Mapp. doi: 10.1002/hbm.24428 – volume: 39 start-page: 3089 issue: 10 year: 2020 ident: 10.1016/j.inffus.2024.102342_b10 article-title: Synthesize high-quality multi-contrast magnetic resonance imaging from multi-echo acquisition using multi-task deep generative model publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2020.2987026 – volume: 38 start-page: 2375 issue: 10 year: 2019 ident: 10.1016/j.inffus.2024.102342_b19 article-title: Image synthesis in multi-contrast MRI with conditional generative adversarial networks publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2019.2901750 – volume: 76 start-page: 355 year: 2021 ident: 10.1016/j.inffus.2024.102342_b2 article-title: A comprehensive survey on multimodal medical signals fusion for smart healthcare systems publication-title: Inf. Fusion doi: 10.1016/j.inffus.2021.06.007 – volume: vol. 30 year: 2017 ident: 10.1016/j.inffus.2024.102342_b26 article-title: Attention is all you need – volume: 83 start-page: 93 year: 2022 ident: 10.1016/j.inffus.2024.102342_b17 article-title: Multi-source information fusion for smart health with artificial intelligence publication-title: Inf. Fusion doi: 10.1016/j.inffus.2022.03.010 – volume: 38 start-page: 1328 issue: 6 year: 2018 ident: 10.1016/j.inffus.2024.102342_b25 article-title: 3D auto-context-based locality adaptive multi-modality GANs for PET synthesis publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2018.2884053 – volume: 38 start-page: 2375 issue: 10 year: 2019 ident: 10.1016/j.inffus.2024.102342_b7 article-title: Image synthesis in multi-contrast MRI with conditional generative adversarial networks publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2019.2901750 – volume: 25 start-page: 565 issue: 5 year: 2019 ident: 10.1016/j.inffus.2024.102342_b15 article-title: Smart healthcare monitoring: A voice pathology detection paradigm for smart cities publication-title: Multimedia Syst. doi: 10.1007/s00530-017-0561-x – year: 2023 ident: 10.1016/j.inffus.2024.102342_b33 article-title: Multi-scale transformer network with edge-aware pre-training for cross-modality MR image synthesis publication-title: IEEE Trans. Med. Imaging – year: 2021 ident: 10.1016/j.inffus.2024.102342_b28 – start-page: 89 year: 2017 ident: 10.1016/j.inffus.2024.102342_b23 article-title: DOTE: Dual convolutional filter learning for super-resolution and cross-modality synthesis in MRI – volume: 93 start-page: 192 year: 2023 ident: 10.1016/j.inffus.2024.102342_b3 article-title: A multi-source information fusion model for outlier detection publication-title: Inf. Fusion doi: 10.1016/j.inffus.2022.12.027 – volume: 76 start-page: 52 year: 2016 ident: 10.1016/j.inffus.2024.102342_b6 article-title: Two-scale image fusion of visible and infrared images using saliency detection publication-title: Infrared Phys. Technol. doi: 10.1016/j.infrared.2016.01.009 – volume: 32 year: 2019 ident: 10.1016/j.inffus.2024.102342_b39 article-title: Deep multimodal multilinear fusion with high-order polynomial pooling publication-title: Adv. Neural Inf. Process. Syst. – start-page: 629 year: 2019 ident: 10.1016/j.inffus.2024.102342_b1 article-title: Deep multi-modal latent representation learning for automated dementia diagnosis – start-page: 127 year: 2016 ident: 10.1016/j.inffus.2024.102342_b5 article-title: Whole image synthesis using a deep encoder-decoder network – volume: 157 year: 2023 ident: 10.1016/j.inffus.2024.102342_b34 article-title: CT synthesis from multi-sequence MRI using adaptive fusion network publication-title: Comput. Biol. Med. doi: 10.1016/j.compbiomed.2023.106738 – volume: 76 start-page: 323 year: 2021 ident: 10.1016/j.inffus.2024.102342_b4 article-title: Image fusion meets deep learning: A survey and perspective publication-title: Inf. Fusion doi: 10.1016/j.inffus.2021.06.008 – start-page: 14 year: 2017 ident: 10.1016/j.inffus.2024.102342_b20 article-title: Deep MR to CT synthesis using unpaired data – volume: 16 start-page: 2639 issue: 12 year: 2004 ident: 10.1016/j.inffus.2024.102342_b36 article-title: Canonical correlation analysis: An overview with application to learning methods publication-title: Neural Comput. doi: 10.1162/0899766042321814 – volume: 33 start-page: 1147 issue: 6 year: 2010 ident: 10.1016/j.inffus.2024.102342_b37 article-title: Multiple kernel learning for dimensionality reduction publication-title: IEEE Trans. Pattern Anal. Mach. Intell. doi: 10.1109/TPAMI.2010.183 – ident: 10.1016/j.inffus.2024.102342_b11 – volume: 39 start-page: 2772 issue: 9 year: 2020 ident: 10.1016/j.inffus.2024.102342_b14 article-title: Hi-net: Hybrid-fusion network for multi-modal MR image synthesis publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2020.2975344 – volume: 38 start-page: 1750 issue: 7 year: 2019 ident: 10.1016/j.inffus.2024.102342_b44 article-title: Ea-GANs: Edge-aware generative adversarial networks for cross-modality MR image synthesis publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2019.2895894 – year: 2021 ident: 10.1016/j.inffus.2024.102342_b43 – volume: 34 start-page: 1993 issue: 10 year: 2014 ident: 10.1016/j.inffus.2024.102342_b42 article-title: The multimodal brain tumor image segmentation benchmark (BRATS) publication-title: IEEE Trans. Med. Imaging doi: 10.1109/TMI.2014.2377694 – start-page: 147 year: 2018 ident: 10.1016/j.inffus.2024.102342_b13 article-title: Generative adversarial training for MRA image synthesis using multi-contrast MRI – volume: 72 start-page: 110 year: 2021 ident: 10.1016/j.inffus.2024.102342_b8 article-title: Image fusion based on generative adversarial network consistent with perception publication-title: Inf. Fusion doi: 10.1016/j.inffus.2021.02.019 – ident: 10.1016/j.inffus.2024.102342_b9 doi: 10.1117/1.JMI.6.1.014005 – volume: 24 start-page: 855 issue: 3 year: 2019 ident: 10.1016/j.inffus.2024.102342_b12 article-title: Bi-modality medical image synthesis using semi-supervised sequential generative adversarial networks publication-title: IEEE J. Biomed. Health Inform. doi: 10.1109/JBHI.2019.2922986 – volume: 6 start-page: 014005 issue: 1 year: 2019 ident: 10.1016/j.inffus.2024.102342_b41 article-title: Fluid-attenuated inversion recovery MRI synthesis from multisequence MRI using three-dimensional fully convolutional networks for multiple sclerosis publication-title: J. Med. Imaging doi: 10.1117/1.JMI.6.1.014005 – ident: 10.1016/j.inffus.2024.102342_b22 doi: 10.1117/1.JMI.6.1.014005 – ident: 10.1016/j.inffus.2024.102342_b21 |
| SSID | ssj0017031 |
| Score | 2.434192 |
| Snippet | Magnetic Resonance Imaging (MRI) is a pivotal neuroimaging technique capable of generating images with various contrasts, known as multi-modal images. The... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 102342 |
| SubjectTerms | Internet of medical things (IoMT) Magnetic resonance imaging (MRI) Multi-modal medical image fusion Smart healthcare |
| Title | Multi-modality MRI fusion with patch complementary pre-training for internet of medical things-based smart healthcare |
| URI | https://dx.doi.org/10.1016/j.inffus.2024.102342 |
| Volume | 107 |
| WOSCitedRecordID | wos001208236800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1872-6305 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017031 issn: 1566-2535 databaseCode: AIEXJ dateStart: 20000701 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3JbtswECXcpIf2UHRF0w089GbQCKn9aAQpmqIOuqSAbwLFBXFgy4YtBcmP9Hs7XETLTZE2h14Eg6BISfM8nBnOPCL0XqbSsGxGpGAyIXGaxySnXJO40CLK-KFU3Er6c3Z6mk-nxZfB4GdXC3M5z-o6v7oqVv9V1NAGwjals3cQdxgUGuA3CB2uIHa4_pPgbUktWSyls7An306GujUxMRdzXYHuPXeZ5C5xfH1tiAJId1aEzTuc2TihasLuu0lJtyd8ErPsyeFmATP7IkqTPNa3cX2Fk0titDOHrJ_r1lWChJYjXxwynbVy6RdRA7_5V37euOOmht_53Gw0jEcBgrCwc8d9MBkZyunFbNGPXrA4ZLoGhZumhCWOsiRoZHcQrtephlzCMXDdUPcu8nBhfBR4n5GZYLTtvsuu_duqF3IRuzS3i9KNUppRSjfKPbTPsqQAhb8_Pjmefgr7U4b13zLx-qfvijJt5uDNp_mz0dMzZM4eo0feA8Fjh5wnaKDqp-jhJND3bp6hdhdDGDCEnSSxwRC2GMI7GMJ9DGEAAO4whJcaewzhPoawxRDeYug5-vHh-OzoI_HncxABjmYDncFg1lmR8yyPMqUOWSUrKkRUpVQyKnMJ7jcVNFGUJ-DXUqrAmdDcECBVKdi1L9BevazVS4SrKBei0qAdNItBq1RpRRV8eCk006lkByjqPmApPHm9ead5eZv4DhAJd60cectf-medbEpvgDrDsgTA3XrnqzvO9Bo92P4b3qC9Zt2qt-i-uGxmm_U7j7Zf8bOtbg |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multi-modality+MRI+fusion+with+patch+complementary+pre-training+for+internet+of+medical+things-based+smart+healthcare&rft.jtitle=Information+fusion&rft.au=Lyu%2C+Jun&rft.au=Chen%2C+Xiudong&rft.au=AlQahtani%2C+Salman+A.&rft.au=Hossain%2C+M.+Shamim&rft.date=2024-07-01&rft.issn=1566-2535&rft.volume=107&rft.spage=102342&rft_id=info:doi/10.1016%2Fj.inffus.2024.102342&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_inffus_2024_102342 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1566-2535&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1566-2535&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1566-2535&client=summon |