Assessment of vaccine information accuracy across large language models.

Background: Use of Large Language Models (LLMs) as a health information source is on the rise. As the amount of people willing to self-diagnose with the tools' help is increasing, verifying the correctness of generated responses becomes extremely important. Methods: Four LLMs (ChatGPT, Copilot, DeepSeek, Gemini) were chosen for the analysis. All models were asked 90 multiple-choice questions on the various topics surrounding vaccination, sourced from a government information website. Additionally, the LLMs were asked two open-ended questions regarding the vaccine information sources, as well as advantages and disadvantages of universal vaccination. Answers were analysed quantitatively and qualitatively. Data was collected in March 2026. Results: Biggest proportion of correct answers was achieved by Gemini (0.98), smallest one - by ChatGPT (0.86). Models' results were found to be non-homogeneous ( Conclusion: Tested LLMs were able to identify correct option when presented with multiple choices and to prepare correct, yet vague information when asked an open-ended question. This points to a limited potential of LLMs serving as a vaccine information source, as the generated content, while factually correct, was not enough to be usable on its own.