Leveraging generative AI to enhance Synthea model development.

Objective: To explore the use of large language models (LLMs) to assist in developing new agent-based disease-specific patient journey models. Materials and Methods: We focus on Synthea, an open-source synthetic health data generator, with the goal of developing models in less time and with reduced expertise, expanding model diversity, and improving synthetic patient data quality. We apply a 4-stage methodology: (1) using an LLM to extract disease information from authoritative medical sources, (2) using an LLM to create an initial Synthea-compatible model, (3) validating that model through 2-level assessment (structural/syntax validation and requirements satisfaction), and (4) using an LLM to iteratively refine the model based on validation feedback. Results: Using hyperthyroidism as an example, we tested Claude 3.5 Sonnet, GPT-4o, and Gemini 1.5 Pro. While the LLMs generated initial models that varied widely in quality, all 3 demonstrated significant improvement in requirement fulfillment scores through successive iterations, with final requirement fulfilment scores approaching 100% for Claude and Gemini. However, evaluation by human experts revealed various structural deficits in final models. Discussion: LLMs can assist in creating patient journey models when combined with structured methodology and authoritative medical knowledge sources. Iterative improvement was shown to be essential in creating models meeting stated requirements. Limitations include frequent medical code inaccuracies, model isolation without comorbidity considerations, and remaining requirements for clinical expertise and human oversight. Conclusion: LLMs can serve as valuable assistive tools for synthetic health data model development when used within structured, iterative frameworks, although at the time of this writing (mid-2024) LLMs require continued human expertise and validation rather than fully autonomous operation. In principle, this conclusion is not limited to Synthea and could be applied to other agent-oriented patient journey frameworks.