ArticleProceedings of the National Academy of Sciences of the United States of America2019
An open challenge to advance probabilistic forecasting for dengue epidemics.
Erratum issuedAbstract read
In one paragraph
Article in Proceedings of the National Academy of Sciences of the United States of America, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 127 papers, 2 of them syntheses that pooled it.
0numbers the graph read from it
0cells of the map it votes in
127citing papers in PubMed, 2 pooled it
–field-weighted citation impact
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
127 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Masks and respirators for prevention of respiratory infections: a state of the science review.Clinical microbiology reviews · 2024Pooled it
- A systematic review of dengue outbreak prediction models: Current scenario and future directions.PLoS neglected tropical diseases · 2023Pooled it
- A real-time forecasting framework for emerging infectious diseases affecting animal populations.PLoS computational biology · 2026Article
- Comparative evaluation of machine learning strategies for short-term dengue forecasting in Brazilian capital municipalities.International journal of biometeorology · 2026Article
- Quantifying, Understanding, and Correcting for Delays in Routine Dengue Case Reporting in the Americas.The American journal of tropical medicine and hygiene · 2026Article
- EpiGIS pro: an AI-powered geospatial intelligence platform for integrated disease surveillance and predictive analytics.International journal of health geographics · 2026Article
- A Software Platform for Collaborative Infectious Disease Modeling.Nature health · 2026Article
- Optimizing wild bird fecal surveillance sites for HPAI through risk estimation using machine learning algorithms.One health (Amsterdam, Netherlands) · 2026Article
- Forecastability of infectious disease time series: are some seasons and pathogens intrinsically more difficult to forecast?PLoS computational biology · 2026Article
- A disease-agnostic approach to ensemble learning for infectious disease forecasting.Nature communications · 2026Article
- Basic Baseline model design choices can substantially influence performance in collaborative forecast hubs.medRxiv : the preprint server for health sciences · 2026Article
- Leveraging probabilistic forecasts for dengue preparedness and control: The 2024 Dengue Forecasting Sprint in Brazil.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Article
- Multi-Model Ensembles in Infectious Disease and Public Health: Methods, Interpretation, and Implementation in R.Statistics in medicine · 2026Article
- Gaussian process emulation for exploring complex infectious disease models.PLoS computational biology · 2025Article
- Climate variation and serotype competition drive dengue outbreak dynamics in Singapore.Nature communications · 2025Article
- Robust uncertainty quantification in popular estimators of the instantaneous reproduction number.American journal of epidemiology · 2025Article
- Multi-model approach to understand and predict past and future dengue epidemic dynamics.Royal Society open science · 2025Article
- A comparative evaluation of multiple machine learning approaches for forecasting dengue outbreaks in Bangladesh.Scientific reports · 2025Article
- Coordinating collaborative infectious disease modeling projects with the hubverse.medRxiv : the preprint server for health sciences · 2025Article
67 more citing papers are in PubMed but not listed here.
Corrections and comments
- Erratum issued
- Erratum issued
Authors and funding
82 authors.
Michael A JohanssonDivision of Vector-Borne Diseases, Centers for Disease Control and Prevention, San Juan 00920, Puerto Rico; mjohansson@cdc.gov.ORCID 0000-0002-5090-7722
Karyn M ApfeldorfData Analytics, Areté Associates, Northridge, CA 91324.
Scott DobsonData Analytics, Areté Associates, Northridge, CA 91324.
Jason DevitaData Analytics, Areté Associates, Northridge, CA 91324.
Anna L BuczakSystems Integration Branch, Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Benjamin BaugherSystems Integration Branch, Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Linda J MonizSystems Integration Branch, Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Thomas BagleySystems Integration Branch, Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Steven M BabinSystems Integration Branch, Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Erhan GuvenSystems Integration Branch, Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Teresa K YamanaDepartment of Environmental Health Sciences, Mailman School of Public Health, Columbia University, New York, NY 10032.
Jeffrey ShamanDepartment of Environmental Health Sciences, Mailman School of Public Health, Columbia University, New York, NY 10032.
Terry MoschouData to Decisions Cooperative Research Center, Kent Town, SA 5067, Australia.
Nick LothianData to Decisions Cooperative Research Center, Kent Town, SA 5067, Australia.
Aaron LaneData to Decisions Cooperative Research Center, Kent Town, SA 5067, Australia.
Grant OsborneData to Decisions Cooperative Research Center, Kent Town, SA 5067, Australia.
Gao JiangHeinz College Information System Management, Carnegie Mellon University, Adelaide, SA 5000, Australia.
Logan C BrooksSchool of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213.
David C FarrowSchool of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213.
Sangwon HyunDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Ryan J TibshiraniSchool of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213.
Roni RosenfeldSchool of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213.
Justin LesslerDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205.ORCID 0000-0002-9741-8109
Nicholas G ReichDepartment of Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0003-3503-9899
Derek A T CummingsDepartment of Biology, University of Florida, Gainesville, FL 32611.ORCID 0000-0002-9437-1907
Stephen A LauerDepartment of Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0003-2948-630X
Sean M MooreDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.
Hannah E ClaphamHospital for Tropical Diseases, Oxford University Clinical Research Unit, Ho Chi Minh City, Vietnam.ORCID 0000-0002-2531-161X
Rachel LoweDepartment of Infectious Disease Epidemiology, London School of Hygiene & Tropical Medicine, London WC1E 7HT, United Kingdom.ORCID 0000-0003-3939-7343
Trevor C BaileyCollege of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4QF, United Kingdom.
Markel García-DíezPredictia Intelligent Data Solutions, 39005 Santander, Spain.
Marilia Sá CarvalhoScientific Computation Program, Oswaldo Cruz Foundation, Rio de Janeiro 21040-900, Brazil.ORCID 0000-0002-9566-0284
Xavier RodóClimate and Health Program, Barcelona Institute for Global Health, 08003 Barcelona, Spain.ORCID 0000-0003-4843-6180
Tridip SardarCatalan Institution for Research and Advanced Studies, 08010 Barcelona, Spain.
Richard PaulDepartment of Mathematical Biology, Indian Statistical Institute, Kolkata, India 700108.
Evan L RayDepartment of Global Health, Centre National de la Recherche Scientifique, 75016 Paris, France.ORCID 0000-0003-4035-0243
Krzysztof SakrejdaDepartment of Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.
Alexandria C BrownDepartment of Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.
Xi MengDepartment of Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0003-2046-9415
Osonde OsobaDepartment of Mathematics and Statistics, Mount Holyoke College, South Hadley, MA 01075.
Raffaele VardavasDepartment of Mathematics and Statistics, Mount Holyoke College, South Hadley, MA 01075.
David ManheimRAND Corporation, Santa MonicaCA 90401.ORCID 0000-0001-8599-8380
Melinda MooreDepartment of Mathematics and Statistics, Mount Holyoke College, South Hadley, MA 01075.
Dhananjai M RaoOpen Philanthropy, San Francisco, CA 94105.
Travis C PorcoDepartment of Computer Science and Software Engineering, Miami University, Oxford, OH 45056.
Sarah AckleyDepartment of Computer Science and Software Engineering, Miami University, Oxford, OH 45056.
Fengchen LiuDepartment of Computer Science and Software Engineering, Miami University, Oxford, OH 45056.
Lee WordenDepartment of Computer Science and Software Engineering, Miami University, Oxford, OH 45056.
Matteo ConvertinoF. I. Proctor Foundation for Research in Ophthalmology, University of California, San Francisco, CA 94122.
Yang LiuInformation Science and Technology, Hokkaido University, Sapporo 060-0808, Japan.
Abraham ReddyInformation Science and Technology, Hokkaido University, Sapporo 060-0808, Japan.
Eloy OrtizDivision of Environmental Health Sciences, School of Public Health, University of Minnesota, Twin Cities, MN 55455.
Jorge RiveroDivision of Environmental Health Sciences, School of Public Health, University of Minnesota, Twin Cities, MN 55455.
Humberto BritoDivision of Environmental Health Sciences, School of Public Health, University of Minnesota, Twin Cities, MN 55455.
Alicia JuarreroDivision of Environmental Health Sciences, School of Public Health, University of Minnesota, Twin Cities, MN 55455.
Leah R JohnsonDepartment of Philosophy, University of Miami, Coral Gables, FL 33146.
Robert B GramacyDepartment of Statistics, Virginia Tech, Blacksburg, VA 24060.
Jeremy M CohenDepartment of Statistics, Virginia Tech, Blacksburg, VA 24060.ORCID 0000-0001-9611-9150
Erin A MordecaiIntegrative Biology, University of South Florida, Tampa, FL 33620.ORCID 0000-0002-4402-5547
Courtney C MurdockDepartment of Biology, Stanford University, Stanford, CA 94305.
Jason R RohrDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.
Sadie J RyanEmerging Pathogens Institute, University of Florida, Gainesville, FL 32611.ORCID 0000-0002-4308-6321
Anna M Stewart-IbarraSchool of Life Sciences, University of KwaZulu, Natal 3629, South Africa.
Daniel P WeikelDepartment of Medicine, State University of New York Upstate Medical University, Syracuse, NY 13421.
Antarpreet JutlaDepartment of Biostatistics, University of Michigan, Ann Arbor, MI 48109.
Rakibul KhanDepartment of Biostatistics, University of Michigan, Ann Arbor, MI 48109.
Marissa PoultneyDepartment of Biostatistics, University of Michigan, Ann Arbor, MI 48109.
Rita R ColwellDepartment of Civil and Environmental Engineering, West Virginia University, Morgantown, WV 26505.
Brenda Rivera-GarcíaDepartment of Cell Biology and Molecular Genetics, University of Maryland, College Park, MD 20742.
Christopher M BarkerPuerto Rico Department of Health, San Juan 00927, Puerto Rico.
Jesse E BellDepartment of Pathology, Microbiology, and Immunology, School of Veterinary Medicine, University of California, Davis, CA 95616.
Matthew BiggerstaffDepartment of Environmental, Agricultural, and Occupational Health, College of Public Health, University of Nebraska Medical Center, Omaha, NE 68198.
David SwerdlowDepartment of Environmental, Agricultural, and Occupational Health, College of Public Health, University of Nebraska Medical Center, Omaha, NE 68198.
Luis Mier-Y-Teran-RomeroDivision of Vector-Borne Diseases, Centers for Disease Control and Prevention, San Juan 00920, Puerto Rico.
Brett M ForsheyInfluenza Division, Centers for Disease Control and Prevention, Atlanta, GA 30329.
Juli TrtanjArmed Forces Health Surveillance Branch, Department of Defense, Silver Spring, MD 20904.
Jason AsherClimate Program Office, National Oceanic and Atmospheric Administration, Silver Spring, MD 20910.
Matt ClayClimate Program Office, National Oceanic and Atmospheric Administration, Silver Spring, MD 20910.
Harold S MargolisDivision of Vector-Borne Diseases, Centers for Disease Control and Prevention, San Juan 00920, Puerto Rico.
Andrew M HebbelerLeidos supporting the Biomedical Advanced Research and Development Authority, Department of Health and Human Services, Washington, DC 20201.
Dylan GeorgeBureau of Oceans, International Environmental and Scientific Affairs, US Department of State, Washington, DC 20520.
Jean-Paul ChretienBureau of Oceans, International Environmental and Scientific Affairs, US Department of State, Washington, DC 20520.ORCID 0000-0001-8143-6823
Funding
Training, Outreach & Diversity ComponentU54GM088491 · NIGMS · UNIVERSITY OF PITTSBURGH AT PITTSBURGH · PI ROBERTS, MARK STENIUS · 2009 to 2018
$16.7MStatistical methods for real-time forecasts of infectious disease: expanding dynamic time-series and machine learning approaches for pandemic scenariosR35GM119582 · NIGMS · UNIVERSITY OF MASSACHUSETTS AMHERST · PI Nicholas G Reich · 2016 to 2026
$4.8MMethods for Reducing Spatial Uncertainty and Bias in Disease SurveillanceR01AI102939 · NIAID · JOHNS HOPKINS UNIVERSITY · PI LESSLER, JUSTIN · 2013 to 2017
$4.6MIntegrated Interdisciplinary, inter-university PhD Program Computational Biology (T32)T32EB009403 · NIBIB · UNIVERSITY OF PITTSBURGH AT PITTSBURGH · PI BAHAR, IVET, BAR-JOSEPH, ZIV · 2009 to 2023
$4.4MLeveraging environmental drivers to predict vector-borne disease transmissionR35GM133439 · NIGMS · STANFORD UNIVERSITY · PI Erin Mordecai · 2019 to 2026
$3.2MDevelopment and Dissemination of Operational Real-Time Respiratory Virus ForecastU01GM110748 · NIGMS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI SHAMAN, JEFFREY L · 2014 to 2018
$2.6M"Modeling contact investigation and rapid response"U01GM087728 · NIGMS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI PORCO, TRAVIS CHRISTIAN · 2011 to 2015
$2.2MInfluenza Outbreak Prediction: Applying Data Assimilation Methodology to Make...R01GM100467 · NIGMS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI SHAMAN, JEFFREY L · 2011 to 2015
$1.4MUS-UK Collab: RCN: Vector Behavior in Transmission Ecology (VectorBiTE)R01AI122284 · NIAID · VIRGINIA POLYTECHNIC INST AND ST UNIV · PI JOHNSON, LEAH RENEE · 2015 to 2019
$520kInference for interacting pathogens with mechanistic and phenomenological modelsR21AI115173 · NIAID · UNIVERSITY OF MASSACHUSETTS AMHERST · PI REICH, NICHOLAS G · 2015 to 2016
$454kHoward Hughes Medical InstituteNIAID NIH HHS R01 AI102939NIAID NIH HHS R01 AI122284NIAID NIH HHS R21 AI115173NIBIB NIH HHS T32 EB009403NIGMS NIH HHS R01 GM100467NIGMS NIH HHS R35 GM119582NIGMS NIH HHS R35 GM133439NIGMS NIH HHS U01 GM087728NIGMS NIH HHS U01 GM110748NIGMS NIH HHS U54 GM088491
Abstract
A wide range of research has promised new tools for forecasting infectious disease dynamics, but little of that research is currently being applied in practice, because tools do not address key public health needs, do not produce probabilistic forecasts, have not been evaluated on external data, or do not provide sufficient forecast skill to be useful. We developed an open collaborative forecasting challenge to assess probabilistic forecasts for seasonal epidemics of dengue, a major global public health problem. Sixteen teams used a variety of methods and data to generate forecasts for 3 epidemiological targets (peak incidence, the week of the peak, and total incidence) over 8 dengue seasons in Iquitos, Peru and San Juan, Puerto Rico. Forecast skill was highly variable across teams and targets. While numerous forecasts showed high skill for midseason situational awareness, early season skill was low, and skill was generally lowest for high incidence seasons, those for which forecasts would be most valuable. A comparison of modeling approaches revealed that average forecast skill was lower for models including biologically meaningful data and mechanisms and that both multimodel and multiteam ensemble forecasts consistently outperformed individual model forecasts. Leveraging these insights, data, and the forecasting framework will be critical to improve forecast skill and the application of forecasts in real time for epidemic preparedness and response. Moreover, key components of this project-integration with public health needs, a common forecasting framework, shared and standardized data, and open participation-can help advance infectious disease forecasting beyond dengue.
Indexed as
Epidemiologic MethodsDengueDisease OutbreaksEpidemicsHumansIncidenceModels, StatisticalPeruPuerto RicodengueepidemicforecastPeruPuerto Rico
Identifiers
What OpenQuestion holds
Textmetadata
LicenceCC BY
Read underepoch 390
Registered trials
None linked
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.