Clinical Testing of an Automated Physicist Chart Checking Tool In Head and Neck Treatment Planning
Abstract
Purpose
Manual medical physicist chart checking is a variable and time-intensive process in external beam radiotherapy QA, especially for complex sites such as head and neck (H&N). This study evaluated the variability and limitations of manual chart checks compared with a custom ESAPI script evaluating 24 parameters.
Methods
To verify script accuracy, it was executed on 20 previously treated H&N plans. One H&N test plan was copied and injected with 13 intentional errors, representing extremes observed in routine planning. Nine clinical physicists independently performed manual checks without knowledge of the errors and without use of the script.
Results
The script found no additional errors in the 20 approved plans. For the intentional error plan, the script correctly identified all occurrences. The average manual check time was 50 minutes. Detection rates varied by error type. All physicists identified prescription and technique errors. Dose and planning-related errors were detected by 50–89% of reviewers. Subtle configuration and structural errors were poorly detected, with only 33–44% identifying incorrect support structures or prescription structure naming discrepancies. None of the reviewers identified an incorrect volume type. Some physicists exhibited search satisfaction bias, concluding their review after detecting a subset of errors. These findings are consistent with prior risk-based QA and incident-learning literature showing configuration and structural definition errors are prone to human oversight. Workload, time pressure, and cognitive expectations may cause physicists to miss clinically relevant errors, supporting TG-275 recommendations for automation in chart-checking.
Conclusion
Automated checks improve efficiency and add a safety layer to manual chart review. If applied earlier, the script would have rapidly identified all injected errors, reduced manual checking time, and prevented errors from reaching the patient, improving treatment quality and safety. We aim to further develop the script and integrate it earlier into the workflow to identify errors sooner.