ArticleJAMIA open2024
Prompt engineering with a large language model to assist providers in responding to patient inquiries: a real-time implementation in the electronic health record.
Article in JAMIA open, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed.
- Generative AI, foundation models and large language models in radiation therapy physics: Clinical applications, challenges, and future directions.Medical physics · 2026Review
- A scoping review of studies on secure messaging through patient portals: persistent challenges and potential solutions.npj health systems · 2026Article
- The Infectious Diseases Orchestrator: Embracing AI Literacy in the Agentic Era.Open forum infectious diseases · 2026Article
- Artificial Intelligence in Medicine: A Definition of Terms.Journal of community hospital internal medicine perspectives · 2026Article
- From algorithms to operating room: can large language models master China's attending anesthesiology exam? A cross-sectional evaluation.International journal of surgery (London, England) · 2026Article
- Development and expert radiologist validation of a custom pipeline for simplification of oncology radiology reports using large language model.Frontiers in oncology · 2026Article
- Leveraging ChatGPT and explainable AI for enhancing clinical decision support.Scientific reports · 2025Article
- Evaluating clinical AI summaries with large language models as judges.NPJ digital medicine · 2025Article
- Prompt-dependent performance of multimodal AI model in oral diagnosis: a comprehensive analysis of accuracy, narrative quality, calibration, and latency versus human experts.Scientific reports · 2025Article
- ChatGPT in Oral Pathology: Bright Promise or Diagnostic Mirage.Medicina (Kaunas, Lithuania) · 2025Article
- A Novel Playbook for Pragmatic Trial Operations to Monitor and Evaluate Ambient Artificial Intelligence in Clinical Practice.NEJM AI · 2025Article
- A Novel Playbook for Pragmatic Trial Operations to Monitor and Evaluate Ambient Artificial Intelligence in Clinical Practice.medRxiv : the preprint server for health sciences · 2025Article
- A systematic review of early evidence on generative AI for drafting responses to patient messages.npj health systems · 2025Article
- Automating Evaluation of AI Text Generation in Healthcare with a Large Language Model (LLM)-as-a-Judge.medRxiv : the preprint server for health sciences · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
Abstract
Background: Large language models (LLMs) can assist providers in drafting responses to patient inquiries. We examined a prompt engineering strategy to draft responses for providers in the electronic health record. The aim was to evaluate the change in usability after prompt engineering. Materials and Methods: A pre-post study over 8 months was conducted across 27 providers. The primary outcome was the provider use of LLM-generated messages from Generative Pre-Trained Transformer 4 (GPT-4) in a mixed-effects model, and the secondary outcome was provider sentiment analysis. Results: Of the 7605 messages generated, 17.5% ( Discussion: The improvement in sentiment with prompt engineering suggests better content quality, but the initial decrease in usage highlights the need for integration with human factors design. Conclusion: Future studies should explore strategies for optimizing the integration of LLMs into the provider workflow to maximize both usability and effectiveness.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.