Evidence map›Paper›PMID 34809631›Full record

Trial reportBMC medical informatics and decision making2021

Improving random forest predictions in small datasets from two-phase sampling designs.

Sunwoo Han, Brian D Williamson, Youyi Fong

Abstract readClinical Trial, Phase III
In one paragraph

Trial report in BMC medical informatics and decision making, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 34 papers.

0numbers the graph read from it
0cells of the map it votes in
34citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

34 citing papers in PubMed.

  1. Trial
  2. Trial
  3. Trial
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Sunwoo HanVaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA.
Brian D WilliamsonVaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA.
Youyi FongVaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA. youyifong@gmail.com.ORCID 0000-0002-4230-6863

Funding

SDMC: HIV Vaccine Trials NetworkUM1AI068635 · NIAID · FRED HUTCHINSON CANCER RESEARCH CENTER · PI Peter B. Gilbert, Yunda Huang · 2011 to 2026
$385.9M
High-Performance Compute Cluster for Comprehensive Cancer and Infectious Diseases ResearchS10OD028685 · OD · FRED HUTCHINSON CANCER RESEARCH CENTER · PI BRADLEY, PHILIP · 2020 to 2020
$2.0M
Statistical methods for HIV-1 immune correlates studiesR01AI122991 · NIAID · FRED HUTCHINSON CANCER RESEARCH CENTER · PI FONG, YOUYI · 2016 to 2020
$1.6M
NIAID NIH HHS R01 AI122991NIAID NIH HHS UM1 AI068635NIH HHS S10 OD028685NIH HHS S10OD028685
6 · The paper itself

Abstract

backgroundWhile random forests are one of the most successful machine learning methods, it is necessary to optimize their performance for use with datasets resulting from a two-phase sampling design with a small number of cases-a common situation in biomedical studies, which often have rare outcomes and covariates whose measurement is resource-intensive.

methodsUsing an immunologic marker dataset from a phase III HIV vaccine efficacy trial, we seek to optimize random forest prediction performance using combinations of variable screening, class balancing, weighting, and hyperparameter tuning.

resultsOur experiments show that while class balancing helps improve random forest prediction performance when variable screening is not applied, class balancing has a negative impact on performance in the presence of variable screening. The impact of the weighting similarly depends on whether variable screening is applied. Hyperparameter tuning is ineffective in situations with small sample sizes. We further show that random forests under-perform generalized linear models for some subsets of markers, and prediction performance on this dataset can be improved by stacking random forests and generalized linear models trained on different subsets of predictors, and that the extent of improvement depends critically on the dissimilarities between candidate learner predictions.

conclusionIn small datasets from two-phase sampling design, variable screening and inverse sampling probability weighting are important for achieving good prediction performance of random forests. In addition, stacking random forests and simple linear models can offer improvements over random forests.

Indexed as

Machine LearningVaccine EfficacyHumansProbabilityCase–control designClass imbalanceHIV vaccineVariable screening

Identifiers

PMID34809631
PMCPMC8607560

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.