Evidence map›Paper›PMID 33485394›Full record

ArticleSystematic reviews2021

Considerations for conducting systematic reviews: evaluating the performance of different methods for de-duplicating references.

Sandra McKeown, Zuhaib M Mir

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Systematic reviews, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 104 papers, 30 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
104citing papers in PubMed, 30 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

104 citing papers in PubMed, 30 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Endoscopic resection versus esophagectomy for T1bN0M0 esophageal cancer: a systematic review and meta-analysis.Diseases of the esophagus : official journal of the International Society for Diseases of the Esophagus · 2026
    Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Interventions on Barriers to the Participation of Adolescents in Physical Activity: A Systematic Review.International journal of environmental research and public health · 2025
    Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. A Systematic Review on Intervention Treatment in Pathological Gambling.International journal of environmental research and public health · 2024
    Pooled it
  19. Pooled it
  20. Pooled it

44 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

2 authors.

Sandra McKeownBracken Health Sciences Library, Queen's University, 18 Stuart Street, Kingston, Ontario, K7L 2V5, Canada. sandra.mckeown@queensu.ca.ORCID 0000-0002-2728-6702
Zuhaib M MirDepartments of Surgery and Public Health Sciences, Queen's University & Kingston Health Sciences Centre, 76 Stuart Street, Kingston, Ontario, K7L 2V7, Canada.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundSystematic reviews involve searching multiple bibliographic databases to identify eligible studies. As this type of evidence synthesis is increasingly pursued, the use of various electronic platforms can help researchers improve the efficiency and quality of their research. We examined the accuracy and efficiency of commonly used electronic methods for flagging and removing duplicate references during this process.

methodsA heterogeneous sample of references was obtained by conducting a similar topical search in MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and PsycINFO databases. References were de-duplicated via manual abstraction to create a benchmark set. The default settings were then used in Ovid multifile search, EndNote desktop, Mendeley, Zotero, Covidence, and Rayyan to de-duplicate the sample of references independently. Using the benchmark set as reference, the number of false-negative and false-positive duplicate references for each method was identified, and accuracy, sensitivity, and specificity were determined.

resultsWe found that the most accurate methods for identifying duplicate references were Ovid, Covidence, and Rayyan. Ovid and Covidence possessed the highest specificity for identifying duplicate references, while Rayyan demonstrated the highest sensitivity.

conclusionThis study reveals the strengths and weaknesses of commonly used de-duplication methods and provides strategies for improving their performance to avoid unintentionally removing eligible studies and introducing bias into systematic reviews. Along with availability, ease-of-use, functionality, and capability, these findings are important to consider when researchers are selecting database platforms and supporting software programs for conducting systematic reviews.

Indexed as

Information Storage and RetrievalSystematic Reviews as TopicDatabases, BibliographicHumansMEDLINEBibliographic databasesDe-duplicationDuplicate referencesReference management softwareStudy designSynthesis methodsSystematic reviewsSystematic review software

Identifiers

PMID33485394
PMCPMC7827976

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.