Physician-Led Electronic Health Record Partnership with AI Drives Accuracy and Efficiency in Hospital Course Summarization

Physician-led partnerships with AI-driven electronic health record (EHR) systems have shown promise in improving the accuracy and efficiency of hospital course summarization. A recent study published in the Journal of the American Medical Association (JAMA) Network Open found that physicians were able to effectively partner with large language models (LLMs) embedded in EHR systems to draft hospital course summaries that were of high quality and concise. The study, conducted by researchers at New York University (NYU) Langone Health, involved a quality improvement study using a convenience sample of 10 internal medicine resident editors and 8 hospitalist evaluators.

Key Takeaways:

• The study found that physician-led partnerships with AI-driven EHR systems can improve the accuracy and efficiency of hospital course summarization, with residents editing a smaller percentage of LLM-generated hospital course summaries (31.5%) compared to physician-generated summaries (44.8%).
• The study concluded that despite the potential influence of artificial time constraints, physician-led partnerships with AI-driven EHR systems are feasible for writing hospital course summaries and can provide a basis for monitoring LLM-generated summaries in clinical practice.
• The study involved a convenience sample of 10 internal medicine resident editors, 8 hospitalist evaluators, and randomly selected general medicine admissions in December 2023 lasting 4 to 8 days at New York University Langone Health.
• The study found that hospitalists compared edited hospital course pairs with A/B testing on the 4Cs (complete, concise, cohesive, and confabulation-free) using 5-point Likert scales converted to 10-point bidirectional scales.

Statistics:

• 100 admissions were involved in the study.
• Residents edited a smaller percentage of LLM-generated hospital course summaries (31.5%) compared to physician-generated summaries (44.8%).
• The semantic change (degree to which the original hospital courses' meaning was altered) after editing was controlled for in the analysis.

Sources:

• NewsRx. Findings from New York University (NYU) Langone Health in the Area of Electronic Medical Records Described (Evaluating Hospital Course Summarization By an Electronic Health Record-based Large Language Model). Information Technology Newsweekly. October 21, 2025; p 215.
• Evaluating Hospital Course Summarization By an Electronic Health Record-based Large Language Model. JAMA Network Open, 2025;8(8).
• American Medical Association (AMA). JAMA Network Open. Contact: 330 N Wabash Ave, Ste 39300, Chicago, IL 60611-5885, USA.