Observational studyJournal of general internal medicine2019
Differences in Narrative Language in Evaluations of Medical Students by Gender and Under-represented Minority Status.
Observational study in Journal of general internal medicine, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 85 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
85 citing papers in PubMed, 1 synthesis or guideline pooled it, 223 citations in OpenAlex.
- Meta-analysis of residency program application and acceptance according to sex, race and ethnicity.The Journal of international medical research · 2024Pooled it
- Adjustment of Assessors' First Impressions Differs by Student Ethnicity.Perspectives on medical education · 2025Trial
- Conflict resolution styles and skills and variation among medical students.BMC medical education · 2023Trial
- A validity evaluation of lexicon-based sentiment analysis of medical students' clinical performance from in-training evaluation reports.Medical education · 2026Article
- Transparency of Alpha Omega Alpha Selection Processes and Bias Mitigation Across US Medical Schools: A Cross-Sectional Mixed Methods Study.Journal of general internal medicine · 2026Article
- Review
- Assessment of Pediatric Residents in the United States by Sex, Ethnoracial and Intersectional Group.Perspectives on medical education · 2026Article
- Two-Tiered Grading in Required Clerkships: Understanding the Why and Results of Making This Change.Cureus · 2026Article
- Does gender influence learning, perceptions and retention in regional anatomy dissection courses?Anatomical science international · 2026Article
- Be AWARE: An Anti-bias Toolkit to Limit Bias in Narrative Evaluation of Medical Trainees.MedEdPORTAL : the journal of teaching and learning resources · 2026Article
- Language Differences by Race in the Narrative Section of the Emergency Medicine Standardized Letter of Evaluation.The western journal of emergency medicine · 2025Article
- Uncovering Disparities in Medical Education: Advisory Guide of Strategies and Best Practices for Illuminating Blind Spots.Medical science educator · 2025Article
- Influence of Race, Ethnicity, and Gender on Clinical Performance Assessments in Graduate Medical Education.Journal of general internal medicine · 2025Article
- Differences in language used to describe racial groups in emergency medicine standardized letter of evaluation.AEM education and training · 2025Article
- Examining Gender-Based Differences in Quantitative Ratings and Narrative Comments in Faculty Assessments by Residents and Fellows.Journal of graduate medical education · 2025Article
- Intersectional Bias and Coded Language in Emergency-Medicine Evaluations: A Response to Gonzalez et al.AEM education and training · 2025Article
- An Analysis of Obstetrics and Gynecology Medical Student Performance Evaluation Clerkship Narratives: Insights From the PRIME+ Framework.Journal of graduate medical education · 2025Article
- Investigating the Road to Equity: A Scoping Review of Solutions to Mitigate Implicit Bias in Assessment within Medical Education.Perspectives on medical education · 2025Article
- Finding the Needle in the Haystack: Can Natural Language Processing of Students' Evaluations of Teachers Identify Teaching Concerns?Journal of general internal medicine · 2025Article
- Characterizing Asian American medical students' experiences with microaggression and the impact on their well-being.Medical education online · 2024Review
25 more citing papers are in PubMed but not listed here.
Corrections and comments
- Commented on by
Authors and funding
8 authors at 3 institutions in 1 country.
Funding
Abstract
backgroundIn varied educational settings, narrative evaluations have revealed systematic and deleterious differences in language describing women and those underrepresented in their fields. In medicine, limited qualitative studies show differences in narrative language by gender and under-represented minority (URM) status.
objectiveTo identify and enumerate text descriptors in a database of medical student evaluations using natural language processing, and identify differences by gender and URM status in descriptions.
designAn observational study of core clerkship evaluations of third-year medical students, including data on student gender, URM status, clerkship grade, and specialty.
participantsA total of 87,922 clerkship evaluations from core clinical rotations at two medical schools in different geographic areas. MAIN MEASURES: We employed natural language processing to identify differences in the text of evaluations for women compared to men and for URM compared to non-URM students. KEY
resultsWe found that of the ten most common words, such as "energetic" and "dependable," none differed by gender or URM status. Of the 37 words that differed by gender, 62% represented personal attributes, such as "lovely" appearing more frequently in evaluations of women (p < 0.001), while 19% represented competency-related behaviors, such as "scientific" appearing more frequently in evaluations of men (p < 0.001). Of the 53 words that differed by URM status, 30% represented personal attributes, such as "pleasant" appearing more frequently in evaluations of URM students (p < 0.001), and 28% represented competency-related behaviors, such as "knowledgeable" appearing more frequently in evaluations of non-URM students (p < 0.001).
conclusionsMany words and phrases reflected students' personal attributes rather than competency-related behaviors, suggesting a gap in implementing competency-based evaluation of students. We observed a significant difference in narrative evaluations associated with gender and URM status, even among students receiving the same grade. This finding raises concern for implicit bias in narrative evaluation, consistent with prior studies, and suggests opportunities for improvement.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.