Evidence map›Paper›PMID 42806083›Full record

ArticleSurgical endoscopy2026

Consensus operational definitions of surgical meta-competencies for computer vision-driven performance assessment: a multi-institutional Delphi study.

Alex H Lee, Aviona Conti, Matthew Leipzig, Chloe K Nobuhara, Reid Dale, Joshua A Villarreal, Charlotte Egeland, Alan Brown, Alfred Song, Brian Sutjiadi and 8 more

Abstract read
PubMed Publisher
In one paragraph

Article in Surgical endoscopy, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

18 authors.

Alex H Lee *Department of Surgery, Stanford University, Stanford, CA, USA.ORCID http://orcid.org/0000-0003-3842-1225
Aviona Conti *Department of Surgery, Johns Hopkins School of Medicine, 1800 Orleans Street, Baltimore, MD, 21287, USA.
Matthew LeipzigDepartment of Cardiothoracic Surgery, Stanford University, Stanford, CA, USA.
Chloe K NobuharaDepartment of Surgery, Stanford University, Stanford, CA, USA.
Reid DaleDepartment of Cardiothoracic Surgery, Stanford University, Stanford, CA, USA.
Joshua A VillarrealDepartment of Surgery, Stanford University, Stanford, CA, USA.
Charlotte EgelandGastrounit Surgical Section, Hvidovre Hospital, Copenhagen University Hospital, Hvidovre, Capital Region, Denmark.
Alan BrownDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.
Alfred SongDepartment of Surgery, Intermountain Health, Salt Lake City, UT, USA.
Brian SutjiadiDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.
Josiah AkliluDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.
Liane S FeldmanDepartment of Surgery, McGill University, Montreal, QC, Canada.
Lisa M KnowltonDepartment of Surgery, Stanford University, Stanford, CA, USA.
Adewale O AdisaDepartment of Surgery, Obafemi Awolowo University, Ife, Osun State, Nigeria.
Dhruv N GhoshDepartment of Paediatric Surgery, Christian Medical College & Hospital, Ludhiana, India.
Ewen M HarrisonUsher Institute, University of Edinburgh, Edinburgh, UK.
Serena Yeung-LevyDepartment of Biomedical Data Science, Stanford University, Stanford, CA, USA.
Jeffrey K JoplingDepartment of Surgery, Johns Hopkins School of Medicine, 1800 Orleans Street, Baltimore, MD, 21287, USA. jopling@jhmi.edu.

Funding

National Science Foundation 2026498
6 · The paper itself

Abstract

backgroundOperative video is increasingly used to evaluate surgical skills. However, existing assessment frameworks rely on subjective constructs or ordinal scales that are poorly suited for consistent annotation or computational analysis. Artificial intelligence (AI) and computer vision can enable automated and scalable performance assessment but require clearly defined, observable performance domains. We aimed to establish expert consensus on video-native surgical meta-competencies with definitions optimized for computer vision-driven assessment of operative performance.

methodsWe conducted an international, two-round modified Delphi study involving surgeons with expertise in surgical education. Five candidate meta-competencies-tissue handling, psychomotor skills, progress, dissection quality, and exposure quality-were defined using binary adequacy-based criteria. Participants rated each definition on discrimination and operationality using five-point Likert scales. Definitions were iteratively refined until consensus (≥ 80%) was achieved.

resultsTwenty experts from 14 institutions across seven countries participated (34% response rate). After Round 2, all five meta-competencies reached consensus, with mean discrimination scores ranging from 4.33 to 4.67 and operationality scores from 4.28 to 4.56. Progress demonstrated the highest discrimination, indicating strong ability to distinguish adequate from inadequate performance, whereas Dissection Quality demonstrated the highest operationality, suggesting that experts considered it the most readily assessable using video. Participants noted that visually identifiable features, such as effective tissue planes, together with binary adequacy criteria, improved confidence in applying the definitions during video review.

conclusionsWe established consensus-based, video-native surgical meta-competencies that provide a standardized framework for computer vision-driven video review. By defining performance using observable constructs designed for consistent annotation across procedural hierarchies, this framework provides a starting point for scalable automated evaluation, pending empirical validation on operative video, to support surgical training and performance assessment.

Indexed as

Artificial intelligenceComputer visionOperative videoSurgical educationTechnical skill assessment

Identifiers

PMID42806083

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.