Evidence map›Paper›PMID 37450264›Full record

ReviewPsychonomic bulletin & review2023

Why many studies of individual differences with inhibition tasks may not localize correlations.

Jeffrey N Rouder, Aakriti Kumar, Julia M Haaf

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in Psychonomic bulletin & review, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 53 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
53citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

53 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Strategy and processing speed eclipse individual differences in control ability in conflict tasks.Journal of experimental psychology. Learning, memory, and cognition · 2022
    Pooled it
  2. Pooled it
  3. Review
  4. Review
  5. Article
  6. Article
  7. Review
  8. Review
  9. Article
  10. Article
  11. Review
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Review
  18. Review
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Jeffrey N RouderUniversity of California, Irvine, CA, USA. jrouder@uci.edu.ORCID http://orcid.org/0000-0003-2023-3891
Aakriti KumarUniversity of California, Irvine, CA, USA.
Julia M HaafUniversity of Amsterdam, Amsterdam, Netherlands.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Individual difference exploration of cognitive domains is predicated on being able to ascertain how well performance on tasks covary. Yet, establishing correlations among common inhibition tasks such as Stroop or flanker tasks has proven quite difficult. It remains unclear whether this difficulty occurs because there truly is a lack of correlation or whether analytic techniques to localize correlations perform poorly real-world contexts because of excessive measurement error from trial noise. In this paper, we explore how well correlations may localized in large data sets with many people, tasks, and replicate trials. Using hierarchical models to separate trial noise from true individual variability, we show that trial noise in 24 extant tasks is about 8 times greater than individual variability. This degree of trial noise results in massive attenuation in correlations and instability in Spearman corrections. We then develop hierarchical models that account for variation across trials, variation across individuals, and covariation across individuals and tasks. These hierarchical models also perform poorly in localizing correlations. The advantage of these models is not in estimation efficiency, but in providing a sense of uncertainty so that researchers are less likely to misinterpret variability in their data. We discuss possible improvements to study designs to help localize correlations.

Indexed as

IndividualityNoiseHumansInhibition, PsychologicalUncertaintyBayesian inferenceCognitive tasksHierarchical modelsIndividual differences

Identifiers

PMID37450264
PMCPMC10728261

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.