ArticleBMJ open2015
A retrospective analysis of the effect of discussion in teleconference and face-to-face scientific peer-review panels.
Article in BMJ open, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
9 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Pooled it
- An observational study of the Patient-Centered Outcomes Research Institute's switch from in-person to virtual research application review panels during the COVID-19 pandemic.Research integrity and peer review · 2026Article
- What works for peer review and decision-making in research funding: a realist synthesis.Research integrity and peer review · 2022Article
- Peer review: Risk and risk tolerance.PloS one · 2022Article
- Clinical trials proposed for the VA Cooperative Studies Program: Success rates and factors impacting approval.Contemporary clinical trials communications · 2021Article
- Article
- A retrospective analysis of the peer review of more than 75,000 Marie Curie proposals between 2007 and 2018.eLife · 2021Article
- Grant reviewer perceptions of the quality, effectiveness, and influence of panel discussion.Research integrity and peer review · 2020Article
- What makes an effective grants peer reviewer? An exploratory study of the necessary skills.PloS one · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveWith the use of teleconferencing for grant peer-review panels increasing, further studies are necessary to determine the efficacy of the teleconference setting compared to the traditional onsite/face-to-face setting. The objective of this analysis was to examine the effects of discussion, namely changes in application scoring premeeting and postdiscussion, in these settings. We also investigated other parameters, including the magnitude of score shifts and application discussion time in face-to-face and teleconference review settings.
designThe investigation involved a retrospective, quantitative analysis of premeeting and postdiscussion scores and discussion times for teleconference and face-to-face review panels. The analysis included 260 and 212 application score data points and 212 and 171 discussion time data points for the face-to-face and teleconference settings, respectively.
resultsThe effect of discussion was found to be small, on average, in both settings. However, discussion was found to be important for at least 10% of applications, regardless of setting, with these applications moving over a potential funding line in either direction (fundable to unfundable or vice versa). Small differences were uncovered relating to the effect of discussion between settings, including a decrease in the magnitude of the effect in the teleconference panels as compared to face-to-face. Discussion time (despite teleconferences having shorter discussions) was observed to have little influence on the magnitude of the effect of discussion. Additionally, panel discussion was found to often result in a poorer score (as opposed to an improvement) when compared to reviewer premeeting scores. This was true regardless of setting or assigned reviewer type (primary or secondary reviewer).
conclusionsSubtle differences were observed between settings, potentially due to reduced engagement in teleconferences. Overall, further research is required on the psychology of decision-making, team performance and persuasion to better elucidate the group dynamics of telephonic and virtual ad-hoc peer-review panels.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.