ArticleMedical education online2026
The hidden patterns of alignment: exploring variability between generative AI and human.
Article in Medical education online, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveAligning medical education curricula with standardized competency-based assessments is a multifaceted task that requires accuracy and consistency. While generative artificial intelligence (GenAI) models offer potential assistance, their performance compared to that of human subject matter experts remains unclear. This study investigates the consistency, accuracy, and strategies of human- and AI-generated alignments of pharmacological topics with the United States Medical Licensing Examination (USMLE) Step 1 content.
methodsA comparative case study examined curricular alignment results produced by GenAI models and human experts. Agreement percentages and pairwise correlation coefficients were calculated to assess consistency and consensus within and between the two groups, highlighting similarities and differences in alignments. We also examined challenges, strategies, and the impact of time constraints on the alignment task.
resultsGenAIs exhibited higher inter-rater reliability in within-group comparisons, with exact agreement percentages ranging from 60% to 79% and stronger correlations. In contrast, human raters demonstrated greater variability, with exact agreement percentages between 14% and 66% and weaker correlations. While partial agreement improved alignment for both groups, GenAIs maintained higher overall consistency and consensus. On the other hand, only two human raters and all GenAI models exhibited high inter-rater reliability in between-group comparisons. GenAIs generated alignments based on contextual analysis and hierarchical mapping, while humans engaged in iterative refinement using their expertise and reflection. Time constraints had a minimal impact on GenAIs, whereas it was acknowledged as a limiting factor for human performance.
conclusionGenAIs can efficiently provide accurate and consistent alignments of pharmacological content in medical curricula but lack the sophisticated decision-making capabilities of human experts. While human alignments show variability due to human experts' diverse perspectives and backgrounds, trained experts make more contextually informed decisions. A conceptual model of Curricular Human-AI Teaming (Curricular HAT) could combine these complementary strengths to enhance the workflow and outcomes of curricular alignment. The model supports the transparency and scalability of human-AI collaboration while maintaining the responsibility of human judgment and ethical use of GenAI.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.