BEST IN PHYSICS (THERAPY): Large Language Model (LLM) Based Linac Downtime Analysis
Abstract
Purpose
Traditional reliability analyses of linear accelerators (LINACs) have typically focused on individual components using limited datasets, failing to exploit the comprehensive operational insights contained within service logs. This study presents a novel, automated framework using a Large Language Model (LLM) to extract and classify maintenance patterns from decade-long historical records at a single institution.
Methods
Our dataset consisted of 1,584 meticulously collected maintenance reports from three Varian TrueBeam LINACs, representing every instance of part failure or repair work over a period of more than ten years. A Python-based Extract-Transform-Load (ETL) pipeline was developed to normalize PDF report formats into a unified structure. A LLM (GPT-4o-mini model), enhanced via few-shot and chain-of-thought prompt engineering, classified reports into 12 failure types and 9 work categories. Model performance was validated against human-annotated ground truth, and time-series decomposition was applied to evaluate downtime trends.
Results
The LLM extracted data was used to gain insights into failure trends, enabling preventative maintenance and supply management which could significantly reduce machine downtime. The LLM demonstrated high classification accuracy, (Failure Type: 0.95 [95% CI: 0.94-0.96]; Work Type: 0.90 [95% CI: 0.88–0.92]). The LLM extracted data was further used to identify specific trends in the maintenance data. Annual downtime consistently remained within the vendor specified 5% downtime threshold, with a single outlier in 2021. The collimation system was identified as the predominant failure source (22%), with component replacement being the most frequent intervention.
Conclusion
This study demonstrates the novel use of LLMs in unifying maintenance report data to enable preventative and proactive maintenance strategies. Our novel method, freely available on GitHub, may enable other clinics to leverage their own data to create proactive maintenance schedules which further minimize operational downtime of LINACs.