Artificial-Intelligence Assisted Structuring of Initial Radiation Oncology Incident Narratives Using AA PM TG-288 Guidelines.
Abstract
Purpose
Only 16% of therapeutic radiation incidents reported nationally to RO-ILS meet “excellent” RO-HAC narrative standards, limiting system learning. Capturing essential context is difficult for time-constrained clinicians, and critical details are often lost before multidisciplinary review. We evaluated whether artificial intelligence (AI), curated with AAPM-TG-288 guidelines, can refine initial narrative reports to facilitate downstream learning.
Methods
Seven clinicians independently extracted TG-288 data elements from our institutional narratives (~80% consensus), informing AI prompt design and mandatory quality checkpoints. A customized AI pipeline with enforced QA was applied to 13 heterogeneous narratives, 5 from our institution, 8 from others (TG-288 examples). The tool distilled free text into seven TG-288 elements (occurrence, relevant circumstances/actions, discovery method, equipment, timespan, recovery actions, individual impact), identified missing content, removed jargon/PHI/conjecture/blame, and generated grammatically corrected narratives. Reporters could amend AI outputs prior to finalization. Our QM committee assessed usability using one common and one unique narrative per reviewer, evaluating ease of use, accuracy, reliability, efficiency, and satisfaction. Inter-rater reliability used Fleiss’ kappa.
Results
Submitter feedback most often addressed AI prompted missing elements including Individual Impact, Relevant Timespan, Method of Discovery, and Recovery Actions. AI produced corrected narratives in 2.35 minutes (average) with 95–98% alignment to available expert assessment. Independent validation of AI classification by 6 QM reviewers showed near-perfect agreement for the common case (κ=0.952), substantial agreement across unique cases with varying quality (κ=0.666), and overall agreement of 89.3% (combined κ=0.636). Reviewers reported a high usability score of 1.43/5 (1=Most favourable).
Conclusion
This work introduces an AI-assisted framework that operationalizes AAPM TG-288 at initial reporting, guiding frontline staff to capture essential context in real time. Embedded quality checks and human feedback improve narrative completeness and interpretability, potentially transforming incident reporting into an active safety intelligence workflow. Limitations include small sample size; broader frontline-staff validation is ongoing.