Automated MRI pituitary structured reporting from free-text using a fine-tuned Llama model: a feasibility study.

BACKGROUND AND

objectiveStructured reports in radiology have demonstrated substantial advantages over unstructured ones. However, the transition from unstructured to structured reporting can face challenges, as experienced radiologists worry about the potential loss of valuable information. In this study, we fine-tuned the Llama 2 model capable of generating structured pituitary MRI reports from unstructured reports.

methodsWe used a training set comprising 104 pituitary MRI reports to fine-tune Llama 2 and 26 reports as a test set to evaluate the system. The dataset was annotated manually by three expert radiologists. For this annotation, the radiologists used the unstructured report and structured it into eight anatomical landmarks: adenohypophysis, pituitary stalk, optic chiasm, suprasellar cistern, neurohypophysis, cavernous sinuses, sphenoid sinuses and other findings.

resultsLlama2 achieves a value greater than 0.79 on the ROUGE-L metric in four anatomical landmarks from free-text pituitary MRI reports. The other anatomical landmarks exceed 0.61 of ROUGE-L except for the other findings section.

conclusionsOur study suggests good performance in structuring anatomical landmarks on pituitary MRI reports using the fine-tune Llama 2 model.