Evidence map›Paper›PMID 34763714›Full record

ArticleTrials2021

Improving peer review of systematic reviews by involving librarians and information specialists: protocol for a randomized controlled trial.

Melissa L Rethlefsen, Sara Schroter, Lex M Bouter, David Moher, Ana Patricia Ayala, Jamie J Kirkham, Maurice P Zeegers

Open access · goldAbstract readClinical Trial Protocol
PubMed Full text at PMC Publisher
In one paragraph

Article in Trials, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
5citing papers in PubMed, 1 pooled it
1.0field-weighted citation impact, top 12% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

5 citing papers in PubMed, 1 synthesis or guideline pooled it, 14 citations in OpenAlex.

  1. Pooled it
  2. Trial
  3. Article
  4. Article
  5. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors at 7 institutions in 4 countries.

Melissa L RethlefsenHealth Sciences Library & Informatics Center, University of New Mexico, MSC 09 5100, 1 University of New Mexico, Albuquerque, NM, 87131-0001, USA. mlrethlefsen@gmail.com.ORCID http://orcid.org/0000-0001-5322-9368
Sara SchroterBMJ Publishing Group, London, England.ORCID https://orcid.org/0000-0002-8791-8564
Lex M BouterDepartment of Philosophy, Faculty of Humanities, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081, HV, Amsterdam, The Netherlands.ORCID https://orcid.org/0000-0002-2659-5482
David MoherCentre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, Centre for Practice Changing Research Building, 501 Smyth Road, PO BOX 201B, Ottawa, Ontario, K1H 8L6, Canada.ORCID https://orcid.org/0000-0003-2434-4206
Ana Patricia AyalaGerstein Science Information Centre, University of Toronto, Toronto, Canada.ORCID https://orcid.org/0000-0002-3613-2270
Jamie J KirkhamCentre for Biostatistics, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.ORCID https://orcid.org/0000-0003-2579-9325
Maurice P ZeegersDepartment of Epidemiology, School for Nutrition and Translational Research in Metabolisms, Care and Health Research Institute, Maastricht University Medical Center+, PO Box 616, 6200, MD, Maastricht, The Netherlands.ORCID https://orcid.org/0000-0002-2387-083X
BMJ Group (United Kingdom)Manchester Academic Health Science Centre · GBOttawa Hospital · CAUniversity Medical Center · USUniversity of New Mexico · USUniversity of Toronto · CAVrije Universiteit Amsterdam · NL

Funding

Medical Research Council MR/S014357/1
6 · The paper itself

Abstract

backgroundProblems continue to exist with the reporting quality and risk of bias in search methods and strategies in systematic reviews and related review types. Peer reviewers who are not familiar with what is required to transparently and fully report a search may not be prepared to review the search components of systematic reviews, nor may they know what is likely to introduce bias into a search. Librarians and information specialists, who have expertise in searching, may offer specialized knowledge that would help improve systematic review search reporting and lessen risk of bias, but they are underutilized as methodological peer reviewers.

methodsThis study will evaluate the effect of adding librarians and information specialists as methodological peer reviewers on the quality of search reporting and risk of bias in systematic review searches. The study will be a pragmatic randomized controlled trial using 150 systematic review manuscripts submitted to BMJ and BMJ Open as the unit of randomization. Manuscripts that report on completed systematic reviews and related review types and have been sent for peer review are eligible. For each manuscript randomized to the intervention, a librarian/information specialist will be invited as an additional peer reviewer using standard practices for each journal. First revision manuscripts will be assessed in duplicate for reporting quality and risk of bias, using adherence to 4 items from PRISMA-S and assessors' judgements on 4 signaling questions from ROBIS Domain 2, respectively. Identifying information from the manuscripts will be removed prior to assessment. DISCUSSION: The primary outcomes for this study are quality of reporting as indicated by differences in the proportion of adequately reported searches in first revision manuscripts between intervention and control groups and risk of bias as indicated by differences in the proportions of first revision manuscripts with high, low, and unclear bias. If the intervention demonstrates an effect on search reporting or bias, this may indicate a need for journal editors to work with librarians and information specialists as methodological peer reviewers.

trial registrationOpen Science Framework. Registered on June 17, 2021, at https://doi.org/10.17605/OSF.IO/W4CK2 .

Indexed as

LibrariansHumansInformation ServicesPeer ReviewPragmatic Clinical Trials as TopicRandomized Controlled Trials as TopicResearch ReportLibrarians and information specialistsLiterature searchingPeer reviewSystematic reviews

Identifiers

PMID34763714
PMCPMC8582191
OpenAlexW3183848989

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.