Investigating Influential Factors of SRS Brain Phantom Delivery Accuracy Using Machine Learning
Abstract
Purpose
The IROC SRS brain phantom provides independent end-to-end tests for clinical trial credentialing; utilizing this unique dataset characterized by heterogeneous community practices, we aimed to evaluate the utility of machine learning in predicting audit outcomes and to investigate risk factors for audit failure.
Methods
We analyzed a dataset of 930 phantom irradiations performed by 673 institutions between 2013 and 2023. The audit passing criteria were: TLD measured dose in the target within ±5% of the calculated dose and 2D gamma passing rate ≥85% using 5%/3mm criteria. The dataset was stratified into seven distinct subgroups based on the machine and TPS category. Random Forest models were trained to predict TLD and gamma values, as well as to classify pass/fail outcomes for each subgroup, using a rigorous 25-repeated 4-fold stratified cross-validation. A total of 60 features included treatment parameters (e.g., collimator type, energy, algorithms), TPS parameters, plan quality metrics (e.g., conformity and gradient), and complexity metrics for modulated plans.
Results
As the largest subgroup (N=320), Varian Eclipse exhibited the second-highest predictive power for TLD dose (MAE~2%), alongside a high sensitivity of 0.98 and a moderate AUC of 0.76. GammaKnife and TomoTherapy models achieved near-perfect classification, albeit with smaller sample sizes (N<80). SHAP analysis identified plan quality metrics (e.g., conformity) as dominant predictors of delivery failure. However, the specific ranking and predictive weight of these features differed by modality; for instance, CI and GI were primary drivers for many subgroups, and complexity contributed significantly for modulated plans, whereas treated target volume metrics played a larger role in dedicated platform predictions.
Conclusion
Machine learning can effectively identify SRS delivery at risk of failure with models tailored to specific delivery modalities. SRS plan quality and the complexity of modulated plans may introduce challenges to accurate dose delivery in independent phantom audits.