Summarize-then-Prompt: A Novel Prompt Engineering Strategy for Generating High-Quality Discharge Summaries.

Accurate discharge summaries are essential for effective communication between hospital and outpatient providers but generating them is labor-intensive. Large language models (LLMs), such as GPT-4, have shown promise in automating this process, potentially reducing clinician workload and improving documentation quality. A recent study using GPT-4 to generate discharge summaries via concatenated clinical notes found that while the summaries were concise and coherent, they often lacked comprehensiveness and contained errors. To address this, we evaluated a structured prompting strategy, summarize-then-prompt, which first generates concise summaries of individual clinical notes before combining them to create a more focused input for the LLM.The objective of this study was to assess the effectiveness of a novel prompting strategy, summarize-then-prompt, in generating discharge summaries that are more complete, accurate, and concise in comparison to the approach that simply concatenates clinical notes.We conducted a retrospective study comparing two prompting strategies: direct concatenation (M1) and summarize-then-prompt (M2). A random sample of 50 hospital stays was selected from a large hospital system. Three attending physicians independently evaluated the generated hospital course summaries for completeness, correctness, and conciseness using a 5-point Likert scale.The summarize-then-prompt strategy outperformed direct concatenation strategy in both completeness (4.28 ± 0.63 vs. 4.01 ± 0.69,