Evidence map›Paper›PMID 39222724›Full record

ArticleJournal of clinical epidemiology2024

A survey of experts to identify methods to detect problematic studies: stage 1 of the INveStigating ProblEmatic Clinical Trials in Systematic Reviews project.

Jack Wilkinson, Calvin Heal, George A Antoniou, Ella Flemyng, Alison Avenell, Virginia Barbour, Esmee M Bordewijk, Nicholas J L Brown, Mike Clarke, Jo Dumville and 22 more

Abstract read
PubMed Publisher
In one paragraph

Article in Journal of clinical epidemiology, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
2citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

2 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

32 authors.

Jack WilkinsonCentre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK. Electronic address: jack.wilkinson@manchester.ac.uk.
Calvin HealCentre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
George A AntoniouManchester Vascular Centre, Manchester University NHS Foundation Trust, Manchester, UK; Division of Cardiovascular Sciences, School of Medical Sciences, Manchester Academic Health Science Centre, The University of Manchester, Manchester, UK.
Ella FlemyngEvidence Production and Methods Directorate, Cochrane Central Executive, London, UK.
Alison AvenellHealth Services Research Unit, University of Aberdeen, Aberdeen, UK.
Virginia BarbourMedical Journal of Australia, Sydney, Australia.
Esmee M BordewijkDepartment of Obstetrics and Gynaecology, Centre for Reproductive Medicine, Amsterdam University Medical Center, Amsterdam, Netherlands.
Nicholas J L BrownDepartment of Psychology, Linnaeus University, Växjö, Sweden.
Mike ClarkeNorthern Ireland Methodology Hub, Queen's University Belfast, Belfast, UK.
Jo DumvilleDivision of Nursing, Midwifery & Social Work, School of Health Sciences, The University of Manchester, Manchester, UK; NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK.
Steph GrohmannEvidence Production and Methods Directorate, Cochrane Central Executive, London, UK.
Lyle C GurrinSchool of Population and Global Health, The University of Melbourne, Melbourne, Australia.
Jill A HaydenDepartment of Community Health & Epidemiology, Dalhousie University, Halifax, Canada.
Kylie E HunterNHMRC Clinical Trials Centre, University of Sydney, Sydney, Australia.
Emily LamIndependent Lay Member, Unaffiliated, Cheshire, UK.
Toby LassersonEvidence Production and Methods Directorate, Cochrane Central Executive, London, UK.
Tianjing LiDepartment of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
Sarah LensenDepartment of Obstetrics, Gynaecology and Newborth Health, Royal Women's Hospital, University of Melbourne, Melbourne, Australia.
Jianping LiuDirector, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China.
Andreas LundhCochrane Denmark & Centre for Evidence-Based Medicine Odense, Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Department of Respiratory Medicine and Infectious Diseases, Copenhagen University Hospital - Bispebjerg and Frederiksberg, Copenhagen, Denmark.
Gideon Meyerowitz-KatzSchool of Health and Society, University of Wollongong, Wollongong, Australia.
Ben W MolDepartment of Obstetrics and Gynaecology, Monash University, Melbourne, Australia.
Neil E O'ConnellDepartment of Health Sciences, Centre for Wellbeing Across the Lifecourse, Brunel University London, London, UK.
Lisa ParkerCharles Perkins Centre, Sydney Medical School, University of Sydney, Sydney, Australia.
Barbara RedmanNew York University, New York, NY, USA.
Anna Lene SeidlerNHMRC Clinical Trials Centre, University of Sydney, Sydney, Australia.
Kyle SheldrickFaculty of Medicine, University of New South Wales, Sydney, Australia.
Emma SydenhamCochrane Central Editorial Service, London, UK.
Darren L DahlyHRB Clinical Research Facility, University College Cork, Cork, Ireland.
Madelon van WelyDepartment of Obstetrics and Gynaecology, Centre for Reproductive Medicine, Amsterdam University Medical Center, Amsterdam, Netherlands.
Lisa BeroUniversity of Colorado Anschutz Medical Campus, Aurora, CO, USA.
Jamie J KirkhamCentre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

BACKGROUND AND

objectiveRandomized controlled trials (RCTs) inform health-care decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesize all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project is developing a tool to identify problematic RCTs in systematic reviews of health care-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion.

methodsWe assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorized these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list.

resultsExtensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool.

conclusionA comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.

Indexed as

Randomized Controlled Trials as TopicSystematic Reviews as TopicHumansResearch DesignSurveys and QuestionnairesCritical appraisalEvidence synthesisFabricationForensic analysisFraudMisconductRandomised controlled trialsResearch integritySystematic reviewsTrustworthiness

Identifiers

PMID39222724

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.