AI-Based Real-Time 3D MRI Predictions from a Single Kv Projection: A Transfer Learning Approach
Abstract
Purpose
Onboard real-time MRI imaging in MRgRT offers exceptional benefits for treating abdominothoracic cancers. However, X-ray based radiotherapy at community clinics cannot provide the same advantages. We developed a deep learning network capable of real-time 3D MRI images from 2D planar kV radiographs. We overcame anatomy changes and the lack of large kV-MRI pair datasets through transfer learning.
Methods
The general workflow trains a baseline model that uses MR projections as input. Then transfer learning sequentially updates the model to predict day-of-treatment anatomy and take kV radiographs as input. A three-phase study was conducted; Phase-1) MRI motion phantom, with scans on a 0.35T MRI and kV projections from a gantry-mounted kV imager. Phase-2) digital anthropomorphic phantom, examining six anatomical changes in stomach size. Phase-3) an IRB-approved retrospective study using patient simulation MRI scans and kV projections acquired during treatment. To get day-of-treatment MRI anatomy, MRI volumes are deformed to CBCT volumes.
Results
With only two respiratory datapoints for anatomy changes, the patient-specific model accurately predicts all respiratory phases in both physical and digital phantom studies. Representative case results are given from each study phase. We list the average DICE score between automated contours from ground truth and AI-predicted images. Phase-1) cuboid insert rotated 60 degrees for anatomy change: 0.986 and 0.966 for static and motion inserts. Phase-2) changes in stomach size from about 300cc to 500cc: 0.920, 0.961, 0.973, and 0.980 for the liver, stomach, left lung, and right lung. Phase-3) preliminary results from the patient study show promise, with 0.916, 0.849, 0.920, and 0.908 for the liver, stomach, left lung, and right lung for one patient.
Conclusion
Transfer learning allows our deep learning model to predict 3D MRI volumes from 2D planar radiographs accurately, paving the way to enable MR-guided radiotherapy on conventional LINACs to benefit community clinics.