Prognostic data extraction harnessing a privacy-preserving large language model: a clinician-AI collaborative retrospective evaluation in head and neck oncology.

Privacy regulations and limited expert-validation constrain the deployment of large language models (LLMs) for electronic health record structuring. We evaluated locally deployed LLMs to extract 30 prognostic variables from 1,360 head and neck cancer reports (882 patients) using zero-shot prompting. A stratified 50-case subset was reviewed by three radiation oncologists (50 cases, 30 fields, 3 reviewers; 4,500 decisions) to form a majority-vote reference for Llama3.3-70B, which achieved 98.6% F1 with high clinician agreement and processed reports in 53 s/report. Among seven additional models (2.6B-70B) benchmarked against this reference, GPT-OSS-20.9B (F1 89.4%) and MedGemma-27B (F1 88.5%) performed best. Integrating LLM-extracted HPV status, smoking history, and Charlson Comorbidity Score into a multivariate Cox Proportional Hazards model (age, sex, T/N stage) improved disease-free survival (likelihood ratio test p = 0.014; ΔC-index + 0.071) and locoregional failure-free survival (p = 0.026; ΔC-index + 0.108) with 1,000-bootstrap internal validation. This clinician-AI collaborative evaluation shows that on-premises LLMs enable privacy-preserving and efficient tumour board support, longitudinal data curation, and outcome prediction.