Evidence map›Paper›PMID 33076832›Full record

ReviewBMC medical research methodology2020

Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance.

Roland Brian Büchter, Alina Weise, Dawid Pieper

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in BMC medical research methodology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 65 papers, 30 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
65citing papers in PubMed, 30 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

65 citing papers in PubMed, 30 syntheses or guidelines pooled it.

  1. Pattern of blast injuries. A Systematic Review: Part 1-Improvised, multiple and unspecified explosive devices.European journal of trauma and emergency surgery : official publication of the European Trauma Society · 2026
    Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

5 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Roland Brian BüchterInstitute for Research in Operative Medicine (IFOM), Faculty of Health - School of Medicine, Witten/Herdecke University, Ostmerheimer Str. 200, 51109, Cologne, Germany. roland.buechter@uni-wh.de.ORCID 0000-0002-2437-4790
Alina WeiseInstitute for Research in Operative Medicine (IFOM), Faculty of Health - School of Medicine, Witten/Herdecke University, Ostmerheimer Str. 200, 51109, Cologne, Germany.
Dawid PieperInstitute for Research in Operative Medicine (IFOM), Faculty of Health - School of Medicine, Witten/Herdecke University, Ostmerheimer Str. 200, 51109, Cologne, Germany.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundData extraction forms link systematic reviews with primary research and provide the foundation for appraising, analysing, summarising and interpreting a body of evidence. This makes their development, pilot testing and use a crucial part of the systematic reviews process. Several studies have shown that data extraction errors are frequent in systematic reviews, especially regarding outcome data.

methodsWe reviewed guidance on the development and pilot testing of data extraction forms and the data extraction process. We reviewed four types of sources: 1) methodological handbooks of systematic review organisations (SRO); 2) textbooks on conducting systematic reviews; 3) method documents from health technology assessment (HTA) agencies and 4) journal articles. HTA documents were retrieved in February 2019 and database searches conducted in December 2019. One author extracted the recommendations and a second author checked them for accuracy. Results are presented descriptively.

resultsOur analysis includes recommendations from 25 documents: 4 SRO handbooks, 11 textbooks, 5 HTA method documents and 5 journal articles. Across these sources the most common recommendations on form development are to use customized or adapted standardised extraction forms (14/25); provide detailed instructions on their use (10/25); ensure clear and consistent coding and response options (9/25); plan in advance which data are needed (9/25); obtain additional data if required (8/25); and link multiple reports of the same study (8/25). The most frequent recommendations on piloting extractions forms are that forms should be piloted on a sample of studies (18/25); and that data extractors should be trained in the use of the forms (7/25). The most frequent recommendations on data extraction are that extraction should be conducted by at least two people (17/25); that independent parallel extraction should be used (11/25); and that procedures to resolve disagreements between data extractors should be in place (14/25).

conclusionsOverall, our results suggest a lack of comprehensiveness of recommendations. This may be particularly problematic for less experienced reviewers. Limitations of our method are the scoping nature of the review and that we did not analyse internal documents of health technology agencies.

Indexed as

Research DesignTechnology Assessment, BiomedicalHumansSystematic Reviews as TopicData extractionEvidence synthesisSystematic review methods

Identifiers

PMID33076832
PMCPMC7574308

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.