Dual-stream pyramid registration network

•We propose a Dual-stream Pyramid Registration Network (referred as Dual-PRNet) for unsupervised 3D medical image registration.•We design a two-stream 3D encoder-decoder network that computes two convolutional feature pyramids separately from two input volumes.•We propose sequential pyramid registra...

Full description

Saved in:
Bibliographic Details
Published in:Medical image analysis Vol. 78; p. 102379
Main Authors: Kang, Miao, Hu, Xiaojun, Huang, Weilin, Scott, Matthew R., Reyes, Mauricio
Format: Journal Article
Language:English
Published: Netherlands Elsevier B.V 01.05.2022
Elsevier BV
Subjects:
3D segmentation
Annotations
Benchmarks
Brain
Brain MRI
Coders
Decoding
Deformable registration
Encoder-decoder network
Encoders-Decoders
Humans
Image Processing, Computer-Assisted
Image registration
Image segmentation
Imaging, Three-Dimensional
Magnetic Resonance Imaging
Medical image registration
Modules
Neural Networks, Computer
Neuroimaging
Pyramids
Registration
Tomography, X-Ray Computed
Warping
3D segmentation
Brain MRI
Encoder-decoder network
Deformable registration
Medical image registration
ISSN:1361-8415, 1361-8423, 1361-8423
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•We propose a Dual-stream Pyramid Registration Network (referred as Dual-PRNet) for unsupervised 3D medical image registration.•We design a two-stream 3D encoder-decoder network that computes two convolutional feature pyramids separately from two input volumes.•We propose sequential pyramid registration where a sequence of pyramid registration (RP) modules is designed to predict multi-level registration fields directly from the decoding feature pyramids.•The PR modules can be further enhanced by computing local 3D correlations between the feature pyramids, resulting in the improved Dual-PRNet++.•Our Dual-PRNet++ can be integrated into a 3D segmentation framework for joint registration and segmentation, by precisely warping voxel-level annotations. [Display omitted] We propose a Dual-stream Pyramid Registration Network (referred as Dual-PRNet) for unsupervised 3D brain image registration. Unlike recent CNN-based registration approaches, such as VoxelMorph, which computes a registration field from a pair of 3D volumes using a single-stream network, we design a two-stream architecture able to estimate multi-level registration fields sequentially from a pair of feature pyramids. Our main contributions are: (i) we design a two-stream 3D encoder-decoder network that computes two convolutional feature pyramids separately from two input volumes; (ii) we propose sequential pyramid registration where a sequence of pyramid registration (PR) modules is designed to predict multi-level registration fields directly from the decoding feature pyramids. The registration fields are refined gradually in a coarse-to-fine manner via sequential warping, which equips the model with a strong capability for handling large deformations; (iii) the PR modules can be further enhanced by computing local 3D correlations between the feature pyramids, resulting in the improved Dual-PRNet++ able to aggregate rich detailed anatomical structure of the brain; (iv) our Dual-PRNet++ can be integrated into a 3D segmentation framework for joint registration and segmentation, by precisely warping voxel-level annotations. Our methods are evaluated on two standard benchmarks for brain MRI registration, where Dual-PRNet++ outperforms the state-of-the-art approaches by a large margin, i.e., improving recent VoxelMorph from 0.511 to 0.748 (Dice score) on the Mindboggle101 dataset. In addition, we further demonstrate that our methods can greatly facilitate the segmentation task in a joint learning framework, by leveraging limited annotations.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1361-8415
1361-8423
1361-8423
DOI:10.1016/j.media.2022.102379