ArticleSurgical endoscopy2012
Is expert peer review obsolete? A model suggests that post-publication reader review may exceed the accuracy of traditional peer review.
Article in Surgical endoscopy, 2012. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed.
- Exploring reviewer self-assessment in the context of academic peer review.Scientific reports · 2025Article
- Do good things really come to those who wait? An analysis of the average time of acceptance in Brazilian surgery journals.Acta cirurgica brasileira · 2024Observational
- Reimagining peer review as an expert elicitation process.BMC research notes · 2022Article
- Keep calm and carry on: moral panic, predatory publishers, peer review, and the emperor's new clothes.Journal of the Medical Library Association : JMLA · 2022Article
- Open up: a survey on open and non-anonymized peer reviewing.Research integrity and peer review · 2020Article
- Guidelines for open peer review implementation.Research integrity and peer review · 2019Article
- Scholarship by the Clinician-Educator in Emergency Medicine.AEM education and training · 2018Article
- What is open peer review? A systematic review.F1000Research · 2017Article
- Reducing Ethical Hazards in Knowledge Production.Science and engineering ethics · 2016Article
- Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study.F1000Research · 2015Article
- Emerging trends in peer review-a survey.Frontiers in neuroscience · 2015Review
- Data publication consensus and controversies.F1000Research · 2014Review
- Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication.ScientometricsArticle
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe peer review process is the gold standard by which academic manuscripts are vetted for publication. However, some investigators have raised concerns regarding its unopposed supremacy, including lack of expediency, susceptibility to editorial bias and statistical limitation due to the small number of reviewers used. Post-publication review-in which the article is assessed by the general readership of the journal instead of a small group of appointed reviewers-could potentially supplement or replace the peer-review process. In this study, we created a computer model to compare the traditional peer-review process to that of post-publication reader review.
methodsWe created a mathematical model of the manuscript review process. A hypothetical manuscript was randomly assigned a "true value" representing its intrinsic quality. We modeled a group of three expert peer reviewers and compared it to modeled groups of 10, 20, 50, or 100 reader-reviewers. Reader-reviewers were assumed to be less skillful at reviewing and were thus modeled to be only ¼ as accurate as expert reviewers. Percentage of correct assessments was calculated for each group.
results400,000 hypothetical manuscripts were modeled. The accuracy of the reader-reviewer group was inferior to the expert reviewer group in the 10-reviewer trial (93.24% correct vs. 97.67%, p < 0.0001) and the 20-reviewer trial (95.50% correct, p < 0.0001). However, the reader-reviewer group surpassed the expert reviewer group in accuracy when 50 or 100 reader-reviewers were used (97.92 and 99.20% respectively, p < 0.0001).
conclusionsIn a mathematical model of the peer review process, the accuracy of public reader-reviewers can surpass that of a small group of expert reviewers if the group of public reviewers is of sufficient size. Further study will be required to determine whether the mathematical assumptions of this model are valid in actual use.
Indexed as
Identifiers
22350231What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.