CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data

•Ultrasound images with annotations are scarce, hampering the application of deep learning methods.•A cross-modal transfer learning strategy is proposed by leveraging annotated data in CT modality.•CycleGAN is adopted to synthesize US images from CT images and construct a transition dataset.•Mainstr...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Ultrasonics Ročník 122; s. 106706
Hlavní autoři: Song, Yuxin, Zheng, Jing, Lei, Long, Ni, Zhipeng, Zhao, Baoliang, Hu, Ying
Médium: Journal Article
Jazyk:angličtina
Vydáno: Netherlands Elsevier B.V 01.05.2022
Témata:
Cross modality
Datasets as Topic
Deep Learning
Humans
Image Processing, Computer-Assisted - methods
Kidney - diagnostic imaging
Kidney segmentation
Tomography, X-Ray Computed
Transfer learning
Ultrasonography
Ultrasound imaging
Cross modality
Transfer learning
Ultrasound imaging
Kidney segmentation
ISSN:0041-624X, 1874-9968, 1874-9968
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:•Ultrasound images with annotations are scarce, hampering the application of deep learning methods.•A cross-modal transfer learning strategy is proposed by leveraging annotated data in CT modality.•CycleGAN is adopted to synthesize US images from CT images and construct a transition dataset.•Mainstream CNN networks such as U-Net, PSPNet and DeepLab v3+ are pretrained on the transition dataset and then transferred to real US images.•The accuracy and generalization ability of ultrasound image segmentation model is effectively improved with limited training data. Accurate segmentation of kidney in ultrasound images is a vital procedure in clinical diagnosis and interventional operation. In recent years, deep learning technology has demonstrated promising prospects in medical image analysis. However, due to the inherent problems of ultrasound images, data with annotations are scarce and arduous to acquire, hampering the application of data-hungry deep learning methods. In this paper, we propose cross-modal transfer learning from computerized tomography (CT) to ultrasound (US) by leveraging annotated data in the CT modality. In particular, we adopt cycle generative adversarial network (CycleGAN) to synthesize US images from CT data and construct a transition dataset to mitigate the immense domain discrepancy between US and CT. Mainstream convolutional neural networks such as U-Net, U-Res, PSPNet, and DeepLab v3+ are pretrained on the transition dataset and then transferred to real US images. We first trained CNN models on a data set composed of 50 ultrasound images and validated them on a validation set composed of 30 ultrasound images. In addition, we selected 82 ultrasound images from another hospital to construct a cross-site data set to verify the generalization performance of the models. The experimental results show that with our proposed transfer learning strategy, the segmentation accuracy in dice similarity coefficient (DSC) reaches 0.853 for U-Net, 0.850 for U-Res, 0.826 for PSPNet and 0.827 for DeepLab v3+ on the cross-site test set. Compared with training from scratch, the accuracy improvement was 0.127, 0.097, 0.105 and 0.036 respectively. Our transfer learning strategy effectively improves the accuracy and generalization ability of ultrasound image segmentation model with limited training data.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0041-624X
1874-9968
1874-9968
DOI:10.1016/j.ultras.2022.106706