SynthesisJournal of graduate medical education2017
Crowdsourcing in Surgical Skills Acquisition: A Developing Technology in Surgical Education.
Synthesis in Journal of graduate medical education, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The use of objective assessments in the evaluation of technical skills in cardiothoracic surgery: a systematic review.Interactive cardiovascular and thoracic surgery · 2022Pooled it
- Objective performance indicators versus GEARS: an opportunity for more accurate assessment of surgical skill.International journal of computer assisted radiology and surgery · 2024Article
- Automated performance metrics and surgical gestures: two methods for assessment of technical skills in robotic surgery.Journal of robotic surgery · 2024Article
- Crowd-sourced and expert video assessment in minimally invasive esophagectomy.Surgical endoscopy · 2023Article
- Utilising an accelerated Delphi process to develop consensus on the requirement and components of a pre-procedural core robotic surgery curriculum.Journal of robotic surgery · 2023Article
- Minimally Trained Analysts Can Perform Fast, Objective Assessment of Orthopedic Technical Skill from Fluoroscopic Images.IISE transactions on healthcare systems engineering · 2022Article
- Crowdsourcing to develop open-access learning resources on antimicrobial resistance.BMC infectious diseases · 2021Article
- Video-Based Coaching: Current Status and Role in Surgical Practice (Part 1) From the Society for Surgery of the Alimentary Tract, Health Care Quality and Outcomes Committee.Journal of gastrointestinal surgery : official journal of the Society for Surgery of the Alimentary Tract · 2021Review
- A Shift in Medical Education During the COVID-19 Pandemic.Advances in experimental medicine and biology · 2021Article
- Robotic dismembered pyeloplasty surgical simulation using a 3D-printed silicone-based model: development, face validation and crowdsourced learning outcomes assessment.Journal of robotic surgery · 2020Article
- Best Approaches to Evaluation and Feedback in Post-Graduate Medical Education.Current urology reports · 2020Review
- A Vision for Using Simulation & Virtual Coaching to Improve the Community Practice of Orthopedic Trauma Surgery.The Iowa orthopaedic journal · 2020Article
- Novel evaluation of surgical activity recognition models using task-based efficiency metrics.International journal of computer assisted radiology and surgery · 2019Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe application of crowdsourcing to surgical education is a recent phenomenon and adds to increasing demands on surgical residency training. The efficacy, range, and scope of this technology for surgical education remains incompletely defined.
objectiveA systematic review was performed using the PubMed database of English-language literature on crowdsourced evaluation of surgical technical tasks up to April 2017.
methodsArticles were reviewed, abstracted, and analyzed, and were assessed for quality using the Medical Education Research Study Quality Instrument (MERSQI). Articles were evaluated with eligibility criteria for inclusion. Study information, performance task, subjects, evaluative standards, crowdworker compensation, time to response, and correlation between crowd and expert or standard evaluations were abstracted and analyzed.
resultsOf 63 unique publications initially identified, 13 with MERSQI scores ranging from 10 to 13 (mean = 11.85) were included in the review. Overall, crowd and expert evaluations demonstrated good to excellent correlation across a wide range of tasks (Pearson's coefficient 0.59-0.95, Cronbach's alpha 0.32-0.92), with 1 exception being a study involving medical students. There was a wide range of reported interrater variability among experts. Nonexpert evaluation was consistently quicker than expert evaluation (ranging from 4.8 to 150.9 times faster), and was more cost effective.
conclusionsCrowdsourced feedback appears to be comparable to expert feedback and is cost effective and efficient. Further work is needed to increase consistency in expert evaluations, to explore sources of discrepant assessments between surgeons and crowds, and to identify optimal populations and novel applications for this technology.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.