ROC-Based Assessment of Clinical Quality Metrics for Iroc Houston SRS Phantom Performance Evaluation
Abstract
Purpose
To evaluate the utility of clinical quality metrics as a means of distinguishing appropriate performance for the IROC Houston stereotactic radiosurgery (SRS) phantom.
Methods
RTOG quality metrics (conformity index (CI), quality of coverage (QoC), homogeneity index (HI) and SRS-specific metrics (Paddick conformity index (PCI), gradient index (GI), R50%, and maximum target dose) were calculated for both planned and irradiated film dose distributions. Receiver Operating Characteristic (ROC) analysis was performed using symmetric threshold sweeps around clinically relevant center values (e.g. CI/PCI/QoC set at 1.0). For each metric, symmetric acceptance windows were systematically widened from ±2% to ±50% in 2% increments, and predictions were compared against IROC phantom pass/fail labels (5% average TLD dose and 85% pass rate at 5%/3 mm criteria for gamma) to compute True Positive Rate (TPR), False Positive Rate (FPR), and Youden’s J statistic. The optimal threshold for each metric was identified as the symmetric window maximizing Youden’s J, providing equivalent acceptance criteria that could complement gamma analysis. ROC curves were generated for both planned and film metric values, allowing comparison of their performance in replicating IROC Houston’s credentialing framework.
Results
ROC analysis shows that Conformity Index (CI) is the best surrogate metric for approximating IROC gamma credentialing decisions, though no single metric fully replicates gamma’s spatial-dose assessment. Optimal thresholds achieving sensitivity like current phantom analysis practice aligns with clinical expectations (i.e. CI between 0.8 - 1.2).
Conclusion
While no single quality metric fully replicates gamma’s spatial-dose assessment, these metrics provide complementary information that can enhance credentialing workflows. CI thresholds aligned with clinical expectations (0.8-1.2) offer a supplementary validation layer that may identify systematic planning issues not apparent from gamma analysis alone. Integration of quality metrics alongside gamma analysis could strengthen credentialing protocols by providing both spatial and scalar perspectives on dose distribution quality.