ArticleBMC medical research methodology2006
An alternative to the hand searching gold standard: validating methodological search filters using relative recall.
Article in BMC medical research methodology, 2006. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 52 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
52 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Search strategies to identify observational studies in MEDLINE and Embase.The Cochrane database of systematic reviews · 2019Pooled it
- Development and validation of a geographic search filter for PubMed for retrieving studies about Nordic countries.Journal of the Medical Library Association : JMLA · 2026Article
- Development and Validation of PubMed and Ovid MEDLINE Search Filters for Exposure Pathways Linking Climate Change With Public Health.Cochrane evidence synthesis and methods · 2026Article
- Evaluating search approaches for a network meta-analysis: a case study of Epistemonikos Database of Trials, CENTRAL, and other sources.BMC medical research methodology · 2026Article
- BibliZap: An exploratory evaluation of an automated multi-level citation searching tool for systematic and rapid reviews.Research synthesis methods · 2026Article
- Development and Validation of a Search Filter to Identify Research on Bisexuality.Archives of sexual behavior · 2026Article
- Using Artificial Intelligence for Text Screening in a Systematic Review of Cardiotoxicity.European urology open science · 2025Article
- Combining search filters for randomized controlled trials with the Cochrane RCT Classifier in Covidence: a methodological validation study.Research synthesis methods · 2025Article
- Translation and validation of a geographic search filter to identify studies about Germany in Embase (Ovid) and MEDLINE(R) ALL (Ovid).Research synthesis methods · 2025Article
- Acute mental health concerns in emergency settings: development and validation of an Ovid MEDLINE search filter.Journal of the Medical Library Association : JMLA · 2025Article
- Development and validation of LGBTQIA+ search filters: report on process and pilot filter for queer women.Journal of the Medical Library Association : JMLA · 2025Article
- A PubMed search filter for efficiently retrieving exercise training studies.BMC medical research methodology · 2024Article
- Use and application of geographical restrictions in systematic reviews with the aim of including studies about Germany: An update of a methodological review.Health information and libraries journal · 2024Review
- The optimal approach for retrieving systematic reviews was achieved when searching MEDLINE and Epistemonikos in addition to reference checking: a methodological validation study.BMC medical research methodology · 2024Article
- Searching for studies: A guide to information retrieval for Campbell systematic reviews.Campbell systematic reviews · 2024Article
- The NICE search filters for treating and managing COVID-19: validation in MEDLINE and Embase (Ovid).Journal of the Medical Library Association : JMLA · 2024Article
- The searchbuildR shiny app: A new implementation of the objective approach for search strategy development in systematic reviews.Cochrane evidence synthesis and methods · 2024Article
- Development and validation of a search strategy and an automated classifier for retrieving temporomandibular disorders studies.Journal of oral & facial pain and headache · 2024Article
- Validation of an interprofessional education search strategy in PubMed to optimize IPE literature searching.Journal of the Medical Library Association : JMLA · 2024Article
- Development and validation of paired MEDLINE and Embase search filters for cost-utility studies.BMC medical research methodology · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundSearch filters or hedges play an important role in evidence-based medicine but their development depends on the availability of a "gold standard"--a reference standard against which to establish the performance of the filter. We demonstrate the feasibility of using relative recall of included studies from multiple systematic reviews to validate methodological search filters as an alternative to validation against a gold standard formed through hand searching.
methodsWe identified 105 Cochrane reviews that used the Highly Sensitive Search Strategy (HSSS), included randomized or quasi-randomized controlled trials, and reported their included studies. We measured the ability of two published and one novel variant of the HSSS to retrieve the MEDLINE-index studies included in these reviews.
resultsThe systematic reviews were comprehensive in their searches. 72% of included primary studies were indexed in MEDLINE. Relative recall of the three strategies ranged from .98 to .91 across all reviews and more comprehensive strategies showed higher recall.
conclusionAn approach using relative recall instead of a hand searching gold standard proved feasible and produced recall figures that were congruent with previously published figures for the HSSS. This technique would permit validation of a methodological filter using a collection of approximately 100 studies of the chosen design drawn from the included studies of multiple systematic reviews that used comprehensive search strategies.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.