ArticleBMC medical research methodology2014
The statistical interpretation of pilot trials: should significance thresholds be reconsidered?
Article in BMC medical research methodology, 2014. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to 5 registered trials, which are not on this map. Cited by 229 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Brain Imaging to Understand HIV-associated Neurocognitive Disorder and Predict Response to Cognitive Training
Engaging Children With Mental Health and Developmental Disabilities and Their Parents to Adopt a Healthy Lifestyle: Piloting the Game Squad Home Exergaming & Virtual Health Coaching Intervention
Effectiveness of Group Versus Individual Self-management of Depression Using a Mobile App to Increase Treatment Adherence Among Users of a University-based Helpline in the Dominican Republic.
In Hospital Administration of Extended-Release Naltrexone for Alcohol Use Disorder: A Pilot Study of Feasibility and Acceptability
Evaluating the Feasibility and Preliminary Effects of a Remotely Delivered Dyad-Based Intervention Incorporating Resistance Training to Improve Fitness in Individuals With Non-Small-Cell Lung Cancer and in Their Partners
Who cites it
229 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Mapping of Dietary Interventions Beneficial in the Prevention of Secondary Health Conditions in Spinal Cord Injured Population: A Systematic Review.The journal of nutrition, health & aging · 2023Pooled it
- Outcomes of a Pilot Mobile App Just-in-Time-Adaptive-Intervention Supporting Coping Skills in Trauma Affected People with HIV.AIDS and behavior · 2026Trial
- Trial
- Testing Implementation Strategies for a Family Healthy Weight Program: Results From the Nebraska Childhood Obesity Research Demonstration 3.0 Pilot Trial.Childhood obesity (Print) · 2026Trial
- Trial
- Study protocol and pilot study results for a clinical intervention trial of PKU carriers and non-carriers: the Phe for Me trial.Orphanet journal of rare diseases · 2026Trial
- Effects of manual diaphragm release on pain, disability and diaphragm function in patients with chronic neck pain: a pilot randomized controlled trial.BMC complementary medicine and therapies · 2025Trial
- Structural brain improvements following individually tailored serious exergame-based training in mild neurocognitive disorders: exploratory randomized controlled trial.Alzheimer's research & therapy · 2025Trial
- HOPE for Hypermobile Ehlers-Danlos Syndrome (hEDS) and Hypermobility Spectrum Disorder (HSD)-A Pilot Randomised Controlled Trial of Feasibility, Acceptability and Appropriateness.European journal of pain (London, England) · 2025Trial
- Feasibility and acceptability of a mobile intervention for patients with psychosis following psychiatric hospitalization: A pilot randomized controlled trial.Psychological services · 2025Trial
- Trial
- Providing an e-cigarette starter kit for smoking cessation and reduction as adjunct to usual care to smokers with a mental health condition: findings from the ESCAPE feasibility study.BMC psychiatry · 2025Trial
- A safety and feasibility randomized placebo controlled trial exploring electroencephalographic effective connectivity neurofeedback treatment for fibromyalgia.Scientific reports · 2025Trial
- Function-based risk reduction intervention for lifestyle-related disorders among inactive 40-year-old people: a pilot randomised controlled trial.BMC public health · 2024Trial
- EMPOWER: A Multi-Site Pilot Trial to Reduce Distress in Surrogate Decision-Makers in the ICU.Journal of pain and symptom management · 2024Trial
- Trial
- Safety and efficacy of sirolimus in hospitalised patients with COVID-19 pneumonia.Respiratory investigation · 2024Trial
- Patient navigation for pregnant individuals with opioid use disorder: Results of a randomized multi-site pilot trial.Addiction (Abingdon, England) · 2024Trial
- Positive psychological interventions on alcohol use and consequences: Pilot randomized trial in a young adult cannabis-using sample.Journal of substance use and addiction treatment · 2024Trial
- Distinct autoantibody profiles across checkpoint inhibitor types and toxicities.Oncoimmunology · 2024Trial
169 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundIn an evaluation of a new health technology, a pilot trial may be undertaken prior to a trial that makes a definitive assessment of benefit. The objective of pilot studies is to provide sufficient evidence that a larger definitive trial can be undertaken and, at times, to provide a preliminary assessment of benefit.
methodsWe describe significance thresholds, confidence intervals and surrogate markers in the context of pilot studies and how Bayesian methods can be used in pilot trials. We use a worked example to illustrate the issues raised.
resultsWe show how significance levels other than the traditional 5% should be considered to provide preliminary evidence for efficacy and how estimation and confidence intervals should be the focus to provide an estimated range of possible treatment effects. We also illustrate how Bayesian methods could also assist in the early assessment of a health technology.
conclusionsWe recommend that in pilot trials the focus should be on descriptive statistics and estimation, using confidence intervals, rather than formal hypothesis testing and that confidence intervals other than 95% confidence intervals, such as 85% or 75%, be used for the estimation. The confidence interval should then be interpreted with regards to the minimum clinically important difference. We also recommend that Bayesian methods be used to assist in the interpretation of pilot trials. Surrogate endpoints can also be used in pilot trials but they must reliably predict the overall effect on the clinical outcome.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.