Comparison of verbal autopsy using a large language model to biologically confirmed causes of death for malaria and other communicable diseases among children in six sub-Saharan African countries.

backgroundMalaria, a preventable parasitic disease, causes most child deaths in sub-Saharan Africa (SSA). Reliable cause-of-death data are essential to evaluate progress toward the national and global malaria control goals. However, civil registration and vital statistics are often weak and incomplete in many low- and middle-income countries. In such circumstances, verbal autopsy (VA) provides an alternative means of mortality surveillance. In some settings, VA has been paired with Minimally Invasive Tissue Sampling (MITS) to obtain detailed biological confirmation of the causes of death. Here, we compare malaria-attributed and all-cause mortality among children younger than five years in six SSA countries, using three computer models (GPT-4o, InSilicoVA, and InterVA-5) to assign causes of death, against MITS as the reference standard.

methodWe examined 3129 under-five deaths enrolled in six Child Health and Mortality Prevention Surveillance (CHAMPS) country sites in SSA between December 2016 and December 2022. Contrived free-text narrative summaries were generated for each record and coded into International Classification of Diseases (ICD-10) codes by GPT-4o. InSilicoVA and InterVA-5 outputs, provided in the World Health Organization 2016 VA codes, were harmonized to ICD-10 for comparison. The primary comparison was the underlying cause of death in VA models and MITS.

resultsSierra Leone had the highest proportion of post-neonatal deaths attributed to malaria at 30.3% (67/221), followed by Kenya at 17.3% (42/243), then Mozambique at 13% (18/138) and Mali at 5.5% (3/55) as defined by MITS. No malaria-attributable deaths were observed in neonates and stillbirths. GPT-4o correctly classified 60 (46.2%) of 130 malaria deaths, compared with 39 (30.0%) for InSilicoVA and 30 (23.1%) for InterVA-5. At the population level, the GPT-4o model achieved a higher cause-specific mortality fraction accuracy (0.36) compared to InSilicoVA (0.07) and InterVA-5 (0.08). GPT-4o performed comparatively better in attributing malaria, HIV/AIDS, and diarrhoeal diseases compared to other communicable diseases.

conclusionGPT-4o demonstrated superior performance over probabilistic VA models in identifying malaria-attributed deaths. National vital registration authorities and health ministries should consider integrating large language model-driven tools into their VA systems to enhance diagnostic precision. While less practicable at scale, focal and periodic MITS comparisons are useful for improving verbal autopsy systems. National mortality data are essential to track progress in reducing childhood deaths from malaria and other conditions.