CNN-based real-time 2D-3D deformable registration from a single X-ray projection

Purpose: The purpose of this paper is to present a method for realtime 2D-3D non-rigid registration using a single fluoroscopic image. Such a method can find applications in surgery, interventional radiology and radiotherapy. By estimating a three-dimensional displacement field from a 2D X-ray image...

Full description

Saved in:
Bibliographic Details
Published in:Journal of clinical medicine Vol. 11; no. 7
Main Authors: Lecomte, Francois, Dillenseger, Jean-Louis, Cotin, Stéphane
Format: Journal Article
Language:English
Published: MDPI 01.03.2024
Subjects:
Computer Science
Image Processing
Image and Video Processing (eess.IV)
FOS: Electrical engineering
2D-3D registration
2D-3D registration deformation deep learning real-time fluoroscopy diffeomorphism
diffeomorphism
deep learning
electronic engineering
Computer Vision and Pattern Recognition (cs.CV)
FOS: Computer and information sciences
real-time
deformation
fluoroscopy
information engineering
ISSN:2077-0383, 2077-0383
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Purpose: The purpose of this paper is to present a method for realtime 2D-3D non-rigid registration using a single fluoroscopic image. Such a method can find applications in surgery, interventional radiology and radiotherapy. By estimating a three-dimensional displacement field from a 2D X-ray image, anatomical structures segmented in the preoperative scan can be projected onto the 2D image, thus providing a mixed reality view. Methods: A dataset composed of displacement fields and 2D projections of the anatomy is generated from the preoperative scan. From this dataset, a neural network is trained to recover the unknown 3D displacement field from a single projection image. Results: Our method is validated on lung 4D CT data at different stages of the lung deformation. The training is performed on a 3D CT using random (non domain-specific) diffeomorphic deformations, to which perturbations mimicking the pose uncertainty are added. The model achieves a mean TRE over a series of landmarks ranging from 2.3 to 5.5 mm depending on the amplitude of deformation. Conclusion: In this paper, a CNN-based method for real-time 2D-3D non-rigid registration is presented. This method is able to cope with pose estimation uncertainties, making it applicable to actual clinical scenarios, such as lung surgery, where the C-arm pose is planned before the intervention.
ISSN:2077-0383
2077-0383
DOI:10.48550/arXiv.2212.07692