Quantitative Risk Assessment In Radiation Oncology Via LLM-Powered Root Cause Analysis of Incident Reports
Abstract
Purpose
Root Cause Analysis (RCA) is resource-intensive and prone to subjective bias. While Radiation Oncology Incident Learning System (RO-ILS) collects vast data, extracting insights from unstructured narratives remains a bottleneck. Large language models (LLMs) offer powerful reasoning capabilities that convert narratives into structured data, enabling quantitative risk assessment to identify systemic risks and vulnerabilities.
Methods
We developed a pipeline using an LLM to perform automated RCA on 254 institutional safety incidents. The LLM classified each incident into the UK Health Security Agency (UKHSA) National Patient Safety Radiotherapy Event Taxonomy, including Contributory Factors and Pathways of Failure. Each incident was assigned a severity score based on TG-100. Quantitative analyses included: Pareto analysis for frequency profiling; Box plots and One-way Analysis of Variance (ANOVA) to assess severity distribution; Ordinal Logistic Regression (OLR) to identify high-risk predictors; and Association Rule Mining (ARM) to visualize latent failure pathways.
Results
Pareto analysis identified "Treatment Delivery" and "Human Behavior" as primary frequency drivers. Significant severity differences were observed across Pathway steps (p < 0.05). Co-occurrence analysis indicated "Procedural" and "Individual factors" were the most prevalent contributory factor categories, with "Department Leadership" and "Physicians" as the most common responsible parties. High-frequency contributory factors included "Process Design" (n=142) and "Slips and Lapses" (n=111). The OLR model identified "Inadequate Leadership" (p < 0.001) and procedural "Violation" (p < 0.001) as significant high-risk predictors of incident severity.
Conclusion
We demonstrated an LLM-powered quantitative risk assessment by converting unstructured incident reports into a structured dataset. This approach enables the identification of key drivers of event severity and the mapping of complex, systemic failure pathways. This methodology paves the way for a new generation of intelligent safety systems to improve the quality and safety of care.