Evidence map›Paper›PMID 39061036›Full record

ArticleBMC medical education2024

Conducting an objective structured clinical examination under COVID-restricted conditions.

Andrea Gotzmann, John Boulet, Yichi Zhang, Judy McCormick, Mathieu Wojcik, Ilona Bartman, Debra Pugh

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMC medical education, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Andrea GotzmannMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada. agotzmann@mcc.ca.
John BouletMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada.
Yichi ZhangMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada.
Judy McCormickMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada.
Mathieu WojcikMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada.
Ilona BartmanMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada.
Debra PughMedical Council of Canada, 1021 Thomas Spratt Place, Ottawa, ON, K1G 5L5, Canada.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundThe administration of performance assessments during the coronavirus disease of 2019 (COVID-19) pandemic posed many challenges, especially for examinations employed as part of certification and licensure. The National Assessment Collaboration (NAC) Examination, an Objective Structured Clinical Examination (OSCE), was modified during the pandemic. The purpose of this study was to gather evidence to support the reliability and validity of the modified NAC Examination.

methodsThe modified NAC Examination was delivered to 2,433 candidates in 2020 and 2021. Cronbach's alpha, decision consistency, and accuracy values were calculated. Validity evidence includes comparisons of scores and sub-scores for demographic groups: gender (male vs. female), type of International Medical Graduate (IMG) (Canadians Studying Abroad (CSA) vs. non-CSA), postgraduate training (PGT) (no PGT vs. PGT), and language of examination (English vs. French). Criterion relationships were summarized using correlations within and between the NAC Examination and the Medical Council of Canada Qualifying Examination (MCCQE) Part I scores.

resultsReliability estimates were consistent with other OSCEs similar in length and previous NAC Examination administrations. Both total score and sub-score differences for gender were statistically significant. Total score differences by type of IMG and PGT were not statistically significant, but sub-score differences were statistically significant. Administration language was not statistically significant for either the total scores or sub-scores. Correlations were all statistically significant with some relationships being small or moderate (0.20 to 0.40) or large (> 0.40).

conclusionsThe NAC Examination yields reliable total scores and pass/fail decisions. Expected differences in total scores and sub-scores for defined groups were consistent with previous literature, and internal relationships amongst NAC Examination sub-scores and their external relationships with the MCCQE Part I supported both discriminant and criterion-related validity arguments. Modifications to OSCEs to address health restrictions can be implemented without compromising the overall quality of the assessment. This study outlines some of the validity and reliability analyses for OSCEs that required modifications due to COVID.

Indexed as

Clinical CompetenceCOVID-19Educational MeasurementCanadaEducation, Medical, GraduateFemaleForeign Medical GraduatesHumansMalePandemicsReproducibility of ResultsSARS-CoV-2Examination disruptionsOSCEPerformance assessmentReliabilityValidity

Identifiers

PMID39061036
PMCPMC11282689

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.