Evidence map›Paper›PMID 32169053›Full record

ReviewBMC medical research methodology2020

Hidden analyses: a review of reporting practice and recommendations for more transparent reporting of initial data analyses.

Marianne Huebner, Werner Vach, Saskia le Cessie, Carsten Oliver Schmidt, Lara Lusa, Topic Group “Initial Data Analysis” of the STRATOS Initiative (STRengthening Analytical Thinking for Observational Studies, http://www.stratos-initiative.org)

Open access · goldAbstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in BMC medical research methodology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 6 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
6citing papers in PubMed, 1 pooled it
1.2field-weighted citation impact, top 18% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

6 citing papers in PubMed, 1 synthesis or guideline pooled it, 15 citations in OpenAlex.

  1. Pooled it
  2. Article
  3. Article
  4. Review
  5. Review
  6. Ten simple rules for initial data analysis.PLoS computational biology · 2022
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors at 4 institutions in 5 countries.

Marianne HuebnerDepartment of Statistics and Probability, Michigan State University, East Lansing, MI, USA. huebner@msu.edu.
Werner VachDepartment of Orthopaedics and Traumatology, University Hospital Basel, Basel, Switzerland.
Saskia le CessieDepartment of Clinical Epidemiology and Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.
Carsten Oliver SchmidtInstitute for Community Medicine, SHIP-KEF University Medicine of Greifswald, Greifswald, Germany.
Lara LusaDepartment of Mathematics, Faculty of Mathematics, Natural Sciences and Information Technology, University of Primorksa, Koper, Slovenia.
Topic Group “Initial Data Analysis” of the STRATOS Initiative (STRengthening Analytical Thinking for Observational Studies, http://www.stratos-initiative.org)
Universität Hamburg · DEUniversity of Primorska · SILeiden University Medical Center · NLUniversitätsmedizin Greifswald · DE

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundIn the data pipeline from the data collection process to the planned statistical analyses, initial data analysis (IDA) typically takes place between the end of the data collection and do not touch the research questions. A systematic process for IDA and clear reporting of the findings would help to understand the potential shortcomings of a dataset, such as missing values, or subgroups with small sample sizes, or shortcomings in the collection process, and to evaluate the impact of these shortcomings on the research results. A clear reporting of findings is also relevant when making datasets available to other researchers. Initial data analyses can provide valuable insights into the suitability of a data set for a future research study. Our aim was to describe the practice of reporting of initial data analyses in observational studies in five highly ranked medical journals with focus on data cleaning, screening, and reporting of findings which led to a potential change in the analysis plan.

methodsThis review was carried out using systematic search strategies with eligibility criteria for articles to be reviewed. A total of 25 papers about observational studies were selected from five medical journals published in 2018. Each paper was reviewed by two reviewers and IDA statements were further discussed by all authors. The consensus was reported.

resultsIDA statements were reported in the methods, results, discussion, and supplement of papers. Ten out of 25 papers (40%) included a statement about data cleaning. Data screening statements were included in all articles, and 18 (72%) indicated the methods used to describe them. Item missingness was reported in 11 papers (44%), unit missingness in 15 papers (60%). Eleven papers (44%) mentioned some changes in the analysis plan. Reported changes referred to missing data treatment, unexpected values, population heterogeneity and aspects related to variable distributions or data properties.

conclusionReporting of initial data analyses were sparse, and statements on IDA were located throughout the research articles. There is a lack of systematic reporting of IDA. We conclude the article with recommendations on how to overcome shortcomings in the practice of IDA reporting in observational studies.

Indexed as

Data AnalysisResearch DesignData CollectionHumansObservational Studies as TopicInitial data analysisObservational studiesReportingSTRATOS initiative

Identifiers

PMID32169053
PMCPMC7071755
OpenAlexW3010807040

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.