Evidence map›Paper›PMID 35346056›Full record

ArticleBMC medical research methodology2022

The effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods.

E Nichols, J A Deal, B K Swenor, A G Abraham, N M Armstrong, K Bandeen-Roche, M C Carlson, M Griswold, F R Lin, T H Mosley and 4 more

Open access · goldAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMC medical research methodology, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.

0numbers the graph read from it
0cells of the map it votes in
5citing papers in PubMed
0.8field-weighted citation impact, top 25% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

5 citing papers in PubMed, 5 citations in OpenAlex.

  1. Observational
  2. Article
  3. Article
  4. Article
  5. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors at 4 institutions in 1 country.

E NicholsDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA. enicho30@jh.edu.
J A DealDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
B K SwenorDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
A G AbrahamDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
N M ArmstrongDepartment of Psychiatry and Human Behavior, Brown University Warren Alpert Medical School, Providence, RI, USA.
K Bandeen-RocheDepartment of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
M C CarlsonDepartment of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
M GriswoldMemory Impairment and Neurodegenerative Dementia Center, University of Mississippi Medical Center, Jackson, MS, USA.
F R LinDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
T H MosleyMemory Impairment and Neurodegenerative Dementia Center, University of Mississippi Medical Center, Jackson, MS, USA.
P Y RamuluWilmer Eye Institute, Johns Hopkins Hospital, Baltimore, MD, USA.
N S ReedDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
A R SharrettDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
A L GrossDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
Johns Hopkins University · USJohns Hopkins Hospital · USUniversity of Mississippi Medical Center · USBrown University · US

Funding

ARIC Neurocognitive Study (ARIC-NCS) Renewal 2023-2028U01HL096812 · NHLBI · NEW YORK UNIVERSITY SCHOOL OF MEDICINE · PI JOSEF CORESH, THOMAS H MOSLEY · 2010 to 2026
$65.7M
Research Education ComponentP30AG066507 · NIA · JOHNS HOPKINS UNIVERSITY · PI Jinchong Xu · 2020 to 2026
$29.3M
ARIC Neurocognitive Study (ARIC-NCS) Renewal 2 of 5U01HL096917 · NHLBI · UNIVERSITY OF MISSISSIPPI MED CTR · PI MOSLEY, THOMAS H · 2010 to 2018
$11.9M
ARIC Neurocognitive Study (ARIC-NCS) Renewal UNC 4 of 5U01HL096899 · NHLBI · UNIV OF NORTH CAROLINA CHAPEL HILL · PI COUPER, DAVID J · 2010 to 2018
$7.5M
ARIC Neurocognitive Study (ARIC-NCS)U01HL096902 · NHLBI · UNIVERSITY OF MINNESOTA · PI LUTSEY, PAMELA L. · 2010 to 2018
$7.4M
Remote Data Enclave Core (RDEC)P30AG066587 · NIA · JOHNS HOPKINS UNIVERSITY · PI DANIEL E POLSKY, Jennifer L. Wolff · 2020 to 2026
$6.7M
ARIC Neurocognitive Study (ARIC-NCS) Renewal 4 of 5U01HL096814 · NHLBI · WAKE FOREST UNIVERSITY HEALTH SCIENCES · PI HUGHES, TIMOTHY M., WAGENKNECHT, LYNNE E · 2010 to 2018
$6.5M
The ARIC and Neurocognitive Longitudinal StudyR01HL070825 · NHLBI · UNIVERSITY OF MISSISSIPPI MEDICAL CENTER · PI MOSLEY, THOMAS H · 2002 to 2004
$5.1M
Vision Loss and Cognition: Testing the sensory consequence hypothesisK01AG052640 · NIA · JOHNS HOPKINS UNIVERSITY · PI SWENOR, BONNIELIN · 2017 to 2021
$834k
Role and Mechanism of Hearing Impairment in Cognitive Decline and DementiaK01AG054693 · NIA · JOHNS HOPKINS UNIVERSITY · PI DEAL, JENNIFER ANNE · 2017 to 2022
$773k
Intersection of Physiologic Frailty and Cognitive Performance in Older Adults: An Integrative Data AnalysisK01AG050699 · NIA · JOHNS HOPKINS UNIVERSITY · PI GROSS, ALDEN L. · 2016 to 2020
$648k
Measurement of Cognitive Function in Older Adults with Sensory LossR21AG060243 · NIA · JOHNS HOPKINS UNIVERSITY · PI DEAL, JENNIFER ANNE, SWENOR, BONNIELIN · 2019 to 2021
$614k
NHLBI NIH HHS HHSN268201700001CNHLBI NIH HHS HHSN268201700001INHLBI NIH HHS HHSN268201700002CNHLBI NIH HHS HHSN268201700002INHLBI NIH HHS HHSN268201700003CNHLBI NIH HHS HHSN268201700003INHLBI NIH HHS HHSN268201700004CNHLBI NIH HHS HHSN268201700004INHLBI NIH HHS HHSN268201700005CNHLBI NIH HHS HHSN268201700005INHLBI NIH HHS R01 HL070825NHLBI NIH HHS U01 HL096812NHLBI NIH HHS U01 HL096814NHLBI NIH HHS U01 HL096899NHLBI NIH HHS U01 HL096902NHLBI NIH HHS U01 HL096917NIA NIH HHS K01 AG050699NIA NIH HHS K01 AG052640NIA NIH HHS K01 AG054693NIA NIH HHS P30 AG066507NIA NIH HHS P30 AG066587NIA NIH HHS R21 AG060243
6 · The paper itself

Abstract

backgroundItem response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.

methodsWe used data from Visit 6 (2016-2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.

resultsWhen missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.

conclusionsIncorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

Indexed as

BiasHumansNeuropsychological TestsCognitionDifferential item functioningItem response theoryMeasurement

Identifiers

PMID35346056
PMCPMC8961895
OpenAlexW4220937040

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.