Comparison of Two Deep-Learning Neural Networks of Single-Image Super-Resolution to Enhance Low-Resolution 3D Cine MR Images for Reconstruction of Time-Resolved 4DMRI
Abstract
Purpose
To enhance the image quality of low-resolution 3D cine magnetic resonance (MR) images acquired at 2Hz by applying two single-image super-resolution deep-learning neural networks, so that the enhanced 3D cine images serve as better templates for more accurate deformable-image-registration-based reconstruction of time-resolved 4DMRI images.
Methods
The Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) and Very-Deep Super-Resolution (VDSR) neural networks were applied to enhance low-resolution 3D cine MR images of 20 subjects. First, the performance of the two models was tested using published optimal hyperparameter sets: The models were trained with 800 pairs of low- and high-resolution 2D photographic images and tested on 100 additional picture pairs. Second, MR image pairs with diaphragm domes within 0-2mm were identified and processed using ImageJ. Patient-specific models were built with 250-400 2D slice pairs and tested with 50-100 slices per subject in three subjects. 5000 iterations, 10 epochs, 0.1 learning rate, 32 patch size, and Adam optimizer were applied to train both models. The enhanced image quality was compared with cubic B-Spline interpolation referenced to the high-resolution ground truth, using Peak-Signal-to-Noise-Ratio (PSNR), Structural Similarity Index (SSIM), Natural Image Quality Evaluator (NIQE), Root-Mean-Square-Error (RMSE), and visual comparison.
Results
The ESRGAN and VDSR models outperform the 3D B-Spline interpolation in both pictures and MR slices. For pictures, VDSR consistently improved in all 4 indexes by 1-5% and visually, but ESRGAN has a significant improvement in NIQE (-39%) and in visual comparison. For MR slices, VDSR enhances images in all 4 indexes and visually by 2.5-12.5%, while SSIM (7.5%), NIQE (-30%), and visual show improvement.
Conclusion
We have demonstrated that ESRGAN is more effective than VDSR in enhancing the quality of 3D cine MR images. Although enhanced 3D cines are useful as TR-4DMRI, they can serve as better templates to produce more accurate time-resolved 4DMRI images.