General versus site specific quantile random forest models for virtual patient-specific quality assurance of VMAT plans.

backgroundModern radiotherapy techniques like volumetric-modulated arc therapy (VMAT) require labor-intensive patient-specific quality assurance (PSQA) measurements, which for busy clinics pose a substantial workflow burden. Most of these measurements simply confirm acceptable plans. This inefficiency has motivated the development of machine learning (ML) models for virtual PSQA to predict the gamma passing rate (GPR) and safely reduce the need for pre-treatment measurements. Plan complexity, often quantified by metrics like the small aperture score (SAS) and average leaf pair opening (ALPO), is a key predictor of PSQA outcome, and its relationship to GPR can vary across different treatment sites. Previous literature exploring ML for PSQA has focused on general models trained on a pooled dataset of all disease sites. PURPOSE: To rigorously compare the predictive performance of a general ML model trained on a pooled dataset of multiple disease sites against site-specific ML models. We hypothesized that isolating plan complexity effects within each disease site would improve predictive accuracy, specifically by reducing false positives and improving specificity.

methodsA total of 15 000 VMAT PSQA measurements (3%/3 mm global criteria, ArcCheck) from multiple disease sites were retrospectively collected. We developed and compared three general models-an XGBoost regressor (XGB), a standard random forest regressor (RFR), and a quantile random forest regressor (QRF)-using 14 plan complexity features as input. Five separate QRF models were also trained for the largest site groups (GI, GU, HN, Lung, and Prostate). For clinical decision-making, the QRF lower-quantile estimate was used as the predicted GPR to ensure a conservative safety margin. Model performance was evaluated using mean absolute error (MAE) and clinical metrics, including sensitivity (correctly identifying passes) and specificity (correctly identifying fails), across various pass/fail thresholds.

resultsThe general QRF model achieved high accuracy (MAE: 0.92%), on par with measurement repeatability. The site-specific QRF models achieved prediction accuracy comparable to or better than the general model. Importantly, they substantially improved specificity and reduced false positives. For example, at the 95% threshold, the general model had an 8.8% false-positive rate, while the prostate-specific and GU-specific models produced (no false positives on the test set (specificity 100%, 19/19 and 5/5 respectively). Because the number of truly failing plans per site is small, these point estimates carry wide 95% confidence intervals (e.g., 82.4%-100% for prostate and 47.8%-100% for GU; Clopper-Pearson). Using a conservative flagged mode, site-specific QRF models reduced false positives to near zero at the cost of reduced but clinically acceptable sensitivity (∼50%-70%). Learning curves showed that the general QRF model plateaued after ∼964 cases, but some site-specific models, like the GU model, required less than 100 cases to reach a stable MAE.

conclusionsTailoring virtual PSQA prediction models to specific disease sites substantially reduces false-positive predictions without compromising overall accuracy. The high specificity of site-specific QRF models strengthens confidence in virtual PSQA and supports clinical implementation that could reduce the number of required QA measurements. The QRF model also provides uncertainty estimates and is highly interpretable, making it a robust tool for enhancing clinical QA efficiency.