Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters.

Objective: To evaluate and compare the quality and safety of ambient digital scribe (ADS) platforms using simulated ambulatory encounters. Methods: Five ADS platforms were evaluated using audio recordings of fourteen simulated clinical encounters. Audio recordings were played on a laptop computer and captured by ADS platforms on a mobile phone. Generated transcripts were compared to professional transcriptions. Clinical notes were graded using rubrics of key elements for each case. Note errors were classified as omission, commission, or partially correct. Potential clinical harm was assessed using the agency for healthcare research and quality harm scale. Note quality was assessed using the 9-item Physician Documentation Quality Instrument (range 9-45). Statistical comparisons included Friedman and χ Results: Transcripts generated by platforms A through D contained an average of 13.9 (95% CI, 6.0-17.5) errors, with 19.5% of the transcript errors transmitted to the clinical note (95% CI, 6.6%-28.8%). For clinical notes, mean percent error across platforms was 26.3% (95% CI, 17.0%-31.0%) with a significantly higher proportion of errors in notes generated by platform E ( Conclusion: Clinical notes generated by ADS platforms using simulated encounters reports important inter-platform and intra-platform variability in accuracy and quality. These findings indicate a need for standardized, objective evaluation and reporting.