ArticleBMC medical research methodology2005
Stability of response characteristics of a Delphi panel: application of bootstrap data expansion.
Article in BMC medical research methodology, 2005. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 386 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
386 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Pooled it
- Internal communication from a happiness management perspective: state-of-the-art and theoretical construction of a guide for its development.BMC psychology · 2024Pooled it
- Establishing a Standardized Clinical Consensus for Reporting Complications Following Lateral Lumbar Interbody Fusion.Medicina (Kaunas, Lithuania) · 2023Guideline
- Developing quality indicators for cross-sectoral psycho-oncology in Germany: combining the RAND/UCLA appropriateness method with a Delphi technique.BMC health services research · 2023Pooled it
- Delphi survey on the most promising areas and methods to improve systematic reviews' production and updating.Systematic reviews · 2023Pooled it
- Co-design of a novel rehabilitation intervention for patients after ankle fracture surgery: establishing healthcare professional consensus.Musculoskeletal surgery · 2025Trial
- Postoperative functional training program for vascularised Iliac flap donor site in jaw defect reconstruction based on the Delphi method.Scientific reports · 2025Trial
- Phase III study of the European Organisation for Research and Treatment of Cancer Quality of Life cancer survivorship core questionnaire.Journal of cancer survivorship : research and practice · 2023Trial
- Consensus Recommendations for Patients with Complex Dispositions: A Modified Delphi Study Among the Strengthening Essential Emergency Departments (SEED).Journal of the American College of Emergency Physicians open · 2026Article
- Development of a nursing-sensitive quality evaluation indicator system for perioperative care in breast reconstruction surgery: A Delphi study.Asia-Pacific journal of oncology nursing · 2026Article
- When does Follow-up on Adverse Events Improve the Understanding of Medicine and Vaccine Safety? Insights from Global Expert Perspectives.Drug safety · 2026Article
- A Consensus-Based Glossary for Controversial Terminology in Root and Canal Anatomy.International endodontic journal · 2026Article
- Constructing the scale to measure financial toxicity in cancer patients for Low-middle income countries (FT-LM) by using the modified Delphi method.Supportive care in cancer : official journal of the Multinational Association of Supportive Care in Cancer · 2026Article
- Developing Language and Image Use Guidance for Communicating About Suicide, Mental Health Concerns and Alcohol and Other Drug Concerns: An Australian Delphi Study.Health promotion journal of Australia : official journal of Australian Association of Health Promotion Professionals · 2026Article
- A modified Delphi consensus on fusion (laparoscopic and robotic) approaches to colorectal surgery in Asia-Pacific.Journal of gastroenterology · 2026Article
- International consensus on dose reduction of biologics for patients with psoriasis: The DR. Delphi study.Journal of the European Academy of Dermatology and Venereology : JEADV · 2026Article
- Management of patients with hypoparathyroidism: a Delphi consensus analysis.Journal of endocrinological investigation · 2026Article
- Expert panel to identify applications of virtual reality (VR) in cognitive behavioral therapy for alcohol use disorder: A Delphi study.Research square · 2026Article
- Article
- Development of a robotic training curriculum for visceral and gastrointestinal surgical trainees: an international Delphi study.Surgical endoscopy · 2026Article
326 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundDelphi surveys with panels of experts in a particular area of interest have been widely utilized in the fields of clinical medicine, nursing practice, medical education and healthcare services. Despite this wide applicability of the Delphi methodology, there is no clear identification of what constitutes a sufficient number of Delphi survey participants to ensure stability of results.
methodsThe study analyzed the response characteristics from the first round of a Delphi survey conducted with 23 experts in healthcare quality and patient safety. The panel members had similar training and subject matter understanding of the Malcolm Baldrige Criteria for Performance Excellence in Healthcare. The raw data from the first round sampling, which usually contains the largest diversity of responses, were augmented via bootstrap sampling to obtain computer-generated results for two larger samples obtained by sampling with replacement. Response characteristics (mean, trimmed mean, standard deviation and 95% confidence intervals) for 54 survey items were compared for the responses of the 23 actual study participants and two computer-generated samples of 1000 and 2000 resampling iterations.
resultsThe results from this study indicate that the response characteristics of a small expert panel in a well-defined knowledge area are stable in light of augmented sampling.
conclusionPanels of similarly trained experts (who possess a general understanding in the field of interest) provide effective and reliable utilization of a small sample from a limited number of experts in a field of study to develop reliable criteria that inform judgment and support effective decision-making.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.