ArticleBMC medical research methodology2020
Searching for Programme theories for a realist evaluation: a case study comparing an academic database search and a simple Google search.
Article in BMC medical research methodology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
11 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Improving communication of prognostic uncertainty in advanced illness: a realist review.BMJ open · 2026Pooled it
- How, why, for whom, and in what contexts organisational and team dynamics shape, and are shaped by, service user researchers in higher education mental health research centres: a realist review protocol.Research involvement and engagement · 2026Review
- Searching smarter, not harder: leveraging AI to enhance literature searches for theory-driven reviews-A methodological case study.BMC medical research methodology · 2026Article
- Understanding integrated mental healthcare for people experiencing forced migration: a realist review protocol.BMJ open · 2026Article
- Integrated mental health for refugees: A realist theory building study.PLOS mental health · 2026Article
- Using Individual Service Funds (ISFs) to Improve Access to Self-Directed Support for Adults With Intellectual Disabilities: A Participatory Realist Review.Journal of applied research in intellectual disabilities : JARID · 2025Review
- Insights from search summary tables for evidence and gap maps: a case study on peer support interventions.Journal of the Medical Library Association : JMLA · 2025Article
- ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer?Clinics and practice · 2024Article
- Searching for studies: A guide to information retrieval for Campbell systematic reviews.Campbell systematic reviews · 2024Article
- Protocol for a scoping review on the development of policy, guidelines and protocols within emergency medical services.British paramedic journal · 2022Article
- Literature searching methods or guidance and their application to public health topics: A narrative review.Health information and libraries journal · 2022Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
backgroundRealist methodologies are increasingly being used to evaluate complex interventions in health and social care. Programme theory (ideas and assumptions of how a particular intervention works) development is the first step in a realist evaluation or a realist synthesis, with literature reviews providing important evidence to support this. Deciding how to search for programme theories is challenging and there is limited guidance available. Using an example of identifying programme theories for a realist evaluation of Pressure Ulcer Risk Assessment Instruments in clinical practice, the authors explore and compare several different approaches to literature searching and highlight important methodological considerations for those embarking on a programme theory review.
methodsWe compared the performance of an academic database search with a simple Google search and developed an optimised search strategy for the identification primary references (i.e. documents providing the clearest examples of programme theories) associated with the use of Pressure Ulcer Risk Assessment Instruments (PU-RAIs). We identified the number of primary references and the total number of references retrieved per source. We then calculated the number needed to read (NNR) expressed as the total number of titles and abstracts screened to identify one relevant reference from each source.
resultsThe academic database search (comprising CINAHL, The Cochrane Library, EMBASE, HMIC, Medline) identified 2 /10 primary references with a NNR of 1395.The Google search identified 7/10 primary references with a NNR of 10.1. The combined NNR was 286.3. The optimised search combining Google and CINAHL identified 10/10 primary references with a NNR of 40.2.
conclusionThe striking difference between the efficiency of the review's academic database and Google searches in finding relevant references prompted an in-depth comparison of the two types of search. The findings indicate the importance of including grey literature sources such as Google in this particular programme theory search, while acknowledging the need for transparency of methods. Further research is needed to facilitate improved guidance for programme theory searches to enhance practice in the realist field and to save researcher time and therefore resource.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.