SynthesisPsychological medicine2015
Is there an excess of significant findings in published studies of psychotherapy for depression?
Synthesis in Psychological medicine, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 24 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
24 citing papers in PubMed, 3 syntheses or guidelines pooled it, 61 citations in OpenAlex.
- Long-term efficacy of psychotherapy for posttraumatic stress disorder: A meta-analysis of randomized controlled trials.Clinical psychology review · 2018Pooled it
- An umbrella review of the literature on the effectiveness of psychological interventions for pain reduction.BMC psychology · 2017Pooled it
- Pooled it
- Randomized Controlled Trial of the Meaning-Making Intervention (MMi) in Patients Newly Diagnosed With Advanced Cancer: Full Trial.Psycho-oncology · 2026Trial
- Treating major depression with yoga: A prospective, randomized, controlled pilot trial.PloS one · 2017Trial
- Implementing open science and reproducible research practices in mental health research through registered reports.JCPP advances · 2024Article
- Editorial Perspective: Misaligned incentives in mental health research - the case for Registered Reports.Journal of child psychology and psychiatry, and allied disciplines · 2024Article
- Psychotherapy and change in mental health-related work disability: a prospective Finnish population-level register-based study with a quasi-experimental design.Journal of epidemiology and community health · 2022Article
- Can pragmatic research, real-world data and digital technologies aid the development of psychedelic medicine?Journal of psychopharmacology (Oxford, England) · 2022Article
- The Cooperative Revolution Reaches Clinical Psychology and Psychotherapy: An Example From Germany.Clinical psychology in Europe · 2021Article
- The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression.Psychological medicine · 2018Article
- Differences in Use of Government Subsidised Mental Health Services by Men and Women with Psychological Distress: A Study of 229,628 Australians Aged 45 Years and Over.Community mental health journal · 2018Article
- Lack of evidence to favor specific preventive interventions in psychosis: a network meta-analysis.World psychiatry : official journal of the World Psychiatric Association (WPA) · 2018Article
- Methylomic profiling of cortex samples from completed suicide cases implicates a role for PSORS1C3 in major depression and suicide.Translational psychiatry · 2017Article
- How to prove that your therapy is effective, even when it is not: a guideline.Epidemiology and psychiatric sciences · 2016Article
- The future of psychotherapy research: stop the waste and focus on issues that matter.Epidemiology and psychiatric sciences · 2016Article
- Evidence-based guidelines for treating bipolar disorder: Revised third edition recommendations from the British Association for Psychopharmacology.Journal of psychopharmacology (Oxford, England) · 2016Article
- Are all psychotherapies equally effective in the treatment of adult depression? The lack of statistical power of comparative outcome studies.Evidence-based mental health · 2016Review
- EPA guidance on improving the image of psychiatry.European archives of psychiatry and clinical neuroscience · 2016Article
- Reporting of Positive Results in Randomized Controlled Trials of Mindfulness-Based Mental Health Interventions.PloS one · 2016Article
Corrections and comments
- Commented on by
Authors and funding
5 authors at 3 institutions in 1 country.
Funding
Abstract
backgroundMany studies have examined the efficacy of psychotherapy for major depressive disorder (MDD) but publication bias against null results may exist in this literature. However, to date, the presence of an excess of significant findings in this literature has not been explicitly tested.
methodWe used a database of 1344 articles on the psychological treatment of depression, identified through systematic search in PubMed, PsycINFO, EMBASE and the Cochrane database of randomized trials. From these we identified 149 studies eligible for inclusion that provided 212 comparisons. We tested for an excess of significant findings using the method developed by Ioannidis and Trikalinos (2007), and compared the distribution of p values in this literature with the distribution in the antidepressant literature, where publication bias is known to be operating.
resultsThe average statistical power to detect the effect size indicated by the meta-analysis was 49%. A total of 123 comparisons (58%) reported a statistically significant difference between treatment and control groups, but on the basis of the average power observed, we would only have expected 104 (i.e. 49%) to do so. There was therefore evidence of an excess of significance in this literature (p = 0.010). Similar results were obtained when these analyses were restricted to studies including a cognitive behavioural therapy (CBT) arm. Finally, the distribution of p values for psychotherapy studies resembled that for published antidepressant studies, where publication bias against null results has already been established.
conclusionsThe small average size of individual psychotherapy studies is only sufficient to detect large effects. Our results indicate an excess of significant findings relative to what would be expected, given the average statistical power of studies of psychotherapy for major depression.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.