ArticleBMC medical research methodology2012
Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm.
Article in BMC medical research methodology, 2012. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
10 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Use of programme theory to understand the differential effects of interventions across socio-economic groups in systematic reviews-a systematic methodology review.Systematic reviews · 2017Pooled it
- Rapid reviews methods series: guidance on rapid scoping, mapping and evidence and gap map ('Big Picture Reviews').BMJ evidence-based medicine · 2025Article
- Developing an Instrument to Evaluate the Quality of Dementia Websites.Healthcare (Basel, Switzerland) · 2023Article
- Consideration of sex and gender in Cochrane reviews of interventions for preventing healthcare-associated infections: a methodology study.BMC health services research · 2019Article
- Best practices in nutrition science to earn and keep the public's trust.The American journal of clinical nutrition · 2019Review
- GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence.Journal of clinical epidemiology · 2017Article
- Consideration of health inequalities in systematic reviews: a mapping review of guidance.Systematic reviews · 2016Review
- Interactive social media interventions to promote health equity: an overview of reviews.Health promotion and chronic disease prevention in Canada : research, policy and practice · 2016Review
- Extending the PRISMA statement to equity-focused systematic reviews (PRISMA-E 2012): explanation and elaboration.International journal for equity in health · 2015Review
- On decreasing inequality in health care in a cost-effective way.BMC health services research · 2014Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
backgroundSystematic reviews have been challenged to consider effects on disadvantaged groups. A priori specification of subgroup analyses is recommended to increase the credibility of these analyses. This study aimed to develop and assess inter-rater agreement for an algorithm for systematic review authors to predict whether differences in effect measures are likely for disadvantaged populations relative to advantaged populations (only relative effect measures were addressed).
methodsA health equity plausibility algorithm was developed using clinimetric methods with three items based on literature review, key informant interviews and methodology studies. The three items dealt with the plausibility of differences in relative effects across sex or socioeconomic status (SES) due to: 1) patient characteristics; 2) intervention delivery (i.e., implementation); and 3) comparators. Thirty-five respondents (consisting of clinicians, methodologists and research users) assessed the likelihood of differences across sex and SES for ten systematic reviews with these questions. We assessed inter-rater reliability using Fleiss multi-rater kappa.
resultsThe proportion agreement was 66% for patient characteristics (95% confidence interval: 61%-71%), 67% for intervention delivery (95% confidence interval: 62% to 72%) and 55% for the comparator (95% confidence interval: 50% to 60%). Inter-rater kappa, assessed with Fleiss kappa, ranged from 0 to 0.199, representing very low agreement beyond chance.
conclusionsUsers of systematic reviews rated that important differences in relative effects across sex and socioeconomic status were plausible for a range of individual and population-level interventions. However, there was very low inter-rater agreement for these assessments. There is an unmet need for discussion of plausibility of differential effects in systematic reviews. Increased consideration of external validity and applicability to different populations and settings is warranted in systematic reviews to meet this need.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.