Evidence map›Paper›PMID 35394862›Full record

ArticleProceedings of the National Academy of Sciences of the United States of America2022

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.

Estee Y Cramer, Evan L Ray, Velma K Lopez, Johannes Bracher, Andrea Brennen, Alvaro J Castro Rivadeneira, Aaron Gerding, Tilmann Gneiting, Katie H House, Yuxin Huang and 285 more

Erratum issuedAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Proceedings of the National Academy of Sciences of the United States of America, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 156 papers.

0numbers the graph read from it
0cells of the map it votes in
156citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

156 citing papers in PubMed.

  1. Article
  2. Review
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Review
  10. Article
  11. Article
  12. Article
  13. Local Influenza Forecasts Outperform State-Level Forecasts in the United States.medRxiv : the preprint server for health sciences · 2026
    Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Leveraging probabilistic forecasts for dengue preparedness and control: The 2024 Dengue Forecasting Sprint in Brazil.Proceedings of the National Academy of Sciences of the United States of America · 2026
    Article

96 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

295 authors.

Estee Y CramerDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0003-1373-3177
Evan L RayDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0003-4035-0243
Velma K LopezCOVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.ORCID 0000-0003-2926-4010
Johannes BracherChair of Econometrics and Statistics, Karlsruhe Institute of Technology, 76185 Karlsruhe, Germany.ORCID 0000-0002-3777-1410
Andrea BrennenIQT Labs, In-Q-Tel, Waltham, MA 02451.
Alvaro J Castro RivadeneiraDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Aaron GerdingDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Tilmann GneitingComputational Statistics Group, Heidelberg Institute for Theoretical Studies, 69118 Heidelberg, Germany.ORCID 0000-0001-9397-3271
Katie H HouseDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Yuxin HuangDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Dasuni JayawardenaDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Abdul H KanjiDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Ayush KhandelwalDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Khoa LeDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Anja MühlemannInstitute of Mathematical Statistics and Actuarial Science, University of Bern, CH-3012 Bern, Switzerland.
Jarad NiemiDepartment of Statistics, Iowa State University, Ames, IA 50011.ORCID 0000-0002-5079-158X
Apurv ShahDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Ariane StarkDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Yijin WangDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Nutcha WattanachitDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Martha W ZornDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.
Youyang GuUnaffiliated, New York, NY 10016.
Sansiddh JainWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Nayana BannurWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Ayush DevaWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Mihir KulkarniWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Srujana MeruguWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Alpan RavalWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Siddhant ShingiWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Avtansh TiwariWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.
Jerome WhiteWadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.ORCID 0000-0003-4148-8834
Neil F AbernethyUniversity of Washington, Seattle, WA 98109.
Spencer WoodyDepartment of Integrative Biology, University of Texas at Austin, Austin, TX 78712.ORCID 0000-0002-2882-3450
Maytal DahanTexas Advanced Computing Center, Austin, TX 78758.
Spencer FoxDepartment of Integrative Biology, University of Texas at Austin, Austin, TX 78712.ORCID 0000-0003-1969-3778
Kelly GaitherTexas Advanced Computing Center, Austin, TX 78758.ORCID 0000-0002-4272-175X
Michael LachmannSanta Fe Institute, Santa Fe, NM 87501.
Lauren Ancel MeyersDepartment of Integrative Biology, University of Texas at Austin, Austin, TX 78712.ORCID 0000-0002-5828-8874
James G ScottDepartment of Information, Risk, and Operations Management, University of Texas at Austin, Austin, TX 78712.
Mauricio TecDepartment of Statistics and Data Sciences, University of Texas at Austin, Austin, TX 78712.ORCID 0000-0002-1853-5842
Ajitesh SrivastavaMing Hsieh Department of Computer and Electrical Engineering, University of Southern California, Los Angeles, CA 90089.
Glover E GeorgeUS Army Engineer Research and Development Center, Vicksburg, MS 39180.ORCID 0000-0003-4779-8702
Jeffrey C CeganUS Army Engineer Research and Development Center, Concord, MA 01742.ORCID 0000-0002-3065-3403
Ian D DettwillerUS Army Engineer Research and Development Center, Vicksburg, MS 39180.
William P EnglandUS Army Engineer Research and Development Center, Vicksburg, MS 39180.
Matthew W FarthingUS Army Engineer Research and Development Center, Vicksburg, MS 39180.
Robert H HunterUS Army Engineer Research and Development Center, Vicksburg, MS 39180.ORCID 0000-0002-2382-7938
Brandon LaffertyUS Army Engineer Research and Development Center, Vicksburg, MS 39180.ORCID 0000-0002-2618-3787
Igor LinkovUS Army Engineer Research and Development Center, Concord, MA 01742.
Michael L MayoUS Army Engineer Research and Development Center, Vicksburg, MS 39180.ORCID 0000-0001-9014-1859
Matthew D ParnoUS Army Engineer Research and Development Center, Hanover, NH 03755.
Michael A RowlandUS Army Engineer Research and Development Center, Vicksburg, MS 39180.ORCID 0000-0002-6759-8225
Benjamin D TrumpUS Army Engineer Research and Development Center, Concord, MA 01742.
Yanli Zhang-JamesDepartment of Psychiatry and Behavioral Sciences, State University of New York Upstate Medical University, Syracuse, NY 13210.
Samuel ChenSchool of Medicine, State University of New York Upstate Medical University, Syracuse, NY 13210.ORCID 0000-0002-1070-9801
Stephen V FaraoneDepartment of Psychiatry and Behavioral Sciences, State University of New York Upstate Medical University, Syracuse, NY 13210.
Jonathan HessDepartment of Psychiatry and Behavioral Sciences, State University of New York Upstate Medical University, Syracuse, NY 13210.
Christopher P MorleyDepartment of Public Health & Preventive Medicine, State University of New York Upstate Medical University, Syracuse, NY 13210.
Asif SalekinDepartment of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13207.ORCID 0000-0002-0807-8967
Dongliang WangDepartment of Public Health & Preventive Medicine, State University of New York Upstate Medical University, Syracuse, NY 13210.
Sabrina M CorsettiDepartment of Physics, University of Michigan, Ann Arbor, MI, 48109.ORCID 0000-0003-2216-2492
Thomas M BaerDepartment of Physics, Trinity University, San Antonio, TX 78212.
Marisa C EisenbergDepartment of Complex Systems, University of Michigan, Ann Arbor, MI 48109.
Karl FalbDepartment of Physics, University of Michigan, Ann Arbor, MI, 48109.ORCID 0000-0002-3465-3988
Yitao HuangDepartment of Physics, University of Michigan, Ann Arbor, MI, 48109.ORCID 0000-0001-7846-2174
Emily T MartinSchool of Public Health, Department of Epidemiology, University of Michigan, Ann Arbor, MI 48109.
Ella McCauleyDepartment of Physics, University of Michigan, Ann Arbor, MI, 48109.
Robert L MyersDepartment of Physics, University of Michigan, Ann Arbor, MI, 48109.
Tom SchwarzDepartment of Physics, University of Michigan, Ann Arbor, MI, 48109.
Daniel SheldonCollege of Information and Computer Sciences, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0002-4257-2432
Graham Casey GibsonSchool of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.
Rose YuDepartment of Computer Science and Engineering, University of California, San Diego, CA 92093.
Liyao GaoDepartment of Statistics, University of Washington, Seattle, WA 98185.
Yian MaHalıcıoğlu Data Science Institute, University of California, San Diego, CA 92093.
Dongxia WuDepartment of Computer Science and Engineering, University of California, San Diego, CA 92093.
Xifeng YanDepartment of Computer Science, University of California, Santa Barbara, CA 93106.
Xiaoyong JinDepartment of Computer Science, University of California, Santa Barbara, CA 93106.
Yu-Xiang WangDepartment of Computer Science, University of California, Santa Barbara, CA 93106.
YangQuan ChenMechatronics, Embedded Systems and Automation Lab, Department of Mechanical Engineering, University of California, Merced, CA 95301.
Lihong GuoJilin University, Changchun City, Jilin Province, 130012, People's Republic of China.ORCID 0000-0003-4804-4005
Yanting ZhaoUniversity of Science and Technology of China, Heifei, Anhui, 230027, People's Republic of China.
Quanquan GuDepartment of Computer Science, University of California, Los Angeles, CA 90095.ORCID 0000-0001-9830-793X
Jinghui ChenDepartment of Computer Science, University of California, Los Angeles, CA 90095.
Lingxiao WangDepartment of Computer Science, University of California, Los Angeles, CA 90095.
Pan XuDepartment of Computer Science, University of California, Los Angeles, CA 90095.ORCID 0000-0002-2559-8622
Weitong ZhangDepartment of Computer Science, University of California, Los Angeles, CA 90095.
Difan ZouDepartment of Computer Science, University of California, Los Angeles, CA 90095.
Hannah BiegelDepartment of Mathematics, University of Arizona, Tucson, AZ 85721.
Joceline LegaDepartment of Mathematics, University of Arizona, Tucson, AZ 85721.ORCID 0000-0003-2064-229X
Steve McConnellConstrux, Bellevue, WA 98004.ORCID 0000-0002-0294-3737
V P NagrajQuality Assurance and Data Science, Signature Science, LLC, Charlottesville, VA 22911.
Stephanie L GuertinQuality Assurance and Data Science, Signature Science, LLC, Charlottesville, VA 22911.
Christopher Hulme-LoweQuality Assurance and Data Science, Signature Science, LLC, Austin, TX 78759.
Stephen D TurnerQuality Assurance and Data Science, Signature Science, LLC, Charlottesville, VA 22911.ORCID 0000-0001-9140-9028
Yunfeng ShiDepartment of Materials Science and Engineering, Rensselaer Polytechnic Institute, Troy, NY 12309.ORCID 0000-0003-1700-6049
Xuegang BanDepartment of Civil and Environmental Engineering, University of Washington, Seattle, WA 98195.
Robert WalravenUnaffiliated, Davis, CA 95616.ORCID 0000-0002-5755-4325
Qi-Jun HongSchool for Engineering of Matter, Transport and Energy, Arizona State University, Tempe, AZ 85287.
Stanley KongManhasset Secondary School, Manhasset, NY 11030.
Axel van de WalleSchool of Engineering, Brown University, Providence, RI 02912.ORCID 0000-0002-3415-1494
James A TurtleInfectious Disease Group, Predictive Science, Inc, San Diego, CA 92121.ORCID 0000-0003-0735-7769
Michal Ben-NunInfectious Disease Group, Predictive Science, Inc, San Diego, CA 92121.ORCID 0000-0002-9164-0008
Steven RileyMedical Research Council Centre for Global Infectious Disease Analysis, Department of Infectious Disease Epidemiology, School of Public Health, Imperial College, W2 1PG London, United Kingdom.ORCID 0000-0001-7904-4804
Pete RileyInfectious Disease Group, Predictive Science, Inc, San Diego, CA 92121.
Ugur KoyluogluFinancial Services, Oliver Wyman, New York, NY 10036.ORCID 0000-0002-6286-351X
David DesRochesOliver Wyman Digital, Oliver Wyman, Boston, MA 02110.
Pedro ForliOliver Wyman Digital, Oliver Wyman, Sao Paolo, Brazil 04711-904.
Bruce HamoryHealth & Life Sciences, Oliver Wyman, Boston, MA 02110.
Christina KyriakidesOliver Wyman Digital, Oliver Wyman, New York, NY 10036.
Helen LeisHealth & Life Sciences, Oliver Wyman, New York, NY 10036.
John MillikenFinancial Services, Oliver Wyman, New York, NY 10036.
Michael MoloneyFinancial Services, Oliver Wyman, New York, NY 10036.
James MorganFinancial Services, Oliver Wyman, New York, NY 10036.
Ninad NirgudkarCore Consultant Group, Oliver Wyman, New York, NY 10036.
Gokce OzcanFinancial Services, Oliver Wyman, New York, NY 10036.
Noah PiwonkaHealth & Life Sciences, Oliver Wyman, New York, NY 10036.
Matt RaviCore Consultant Group, Oliver Wyman, New York, NY 10036.
Chris SchraderHealth & Life Sciences, Oliver Wyman, New York, NY 10036.
Elizabeth ShakhnovichHealth & Life Sciences, Oliver Wyman, New York, NY 10036.
Daniel SiegelFinancial Services, Oliver Wyman, New York, NY 10036.
Ryan SpatzCore Consultant Group, Oliver Wyman, New York, NY 10036.
Chris StiefelingFinancial Services, Oliver Wyman Digital, Toronto, ON, Canada M5J 0A1.
Barrie WilkinsonFinancial Services, Oliver Wyman, London, UK W1U 8EW.
Alexander WongOliver Wyman Digital, Oliver Wyman, New York, NY 10036.
Sean CavanyDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.ORCID 0000-0002-2559-797X
Guido EspañaDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.ORCID 0000-0002-9915-8056
Sean MooreDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.ORCID 0000-0001-9062-6100
Rachel OidtmanDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.ORCID 0000-0003-1773-9533
Alex PerkinsDepartment of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.ORCID 0000-0002-7518-4014
David KrausDepartment of Mathematics and Statistics, Masaryk University, 61137 Brno, Czech Republic.ORCID 0000-0003-4376-3932
Andrea KrausDepartment of Mathematics and Statistics, Masaryk University, 61137 Brno, Czech Republic.
Zhifeng GaoMicrosoft, Redmond, WA 98029.
Jiang BianMicrosoft, Redmond, WA 98029.
Wei CaoMicrosoft, Redmond, WA 98029.ORCID 0000-0001-5640-0917
Juan Lavista FerresMicrosoft, Redmond, WA 98029.ORCID 0000-0002-9654-3178
Chaozhuo LiMicrosoft, Redmond, WA 98029.
Tie-Yan LiuMicrosoft, Redmond, WA 98029.
Xing XieMicrosoft, Redmond, WA 98029.
Shun ZhangMicrosoft, Redmond, WA 98029.
Shun ZhengMicrosoft, Redmond, WA 98029.
Alessandro VespignaniInstitute for Scientific Interchange Foundation, Turin, 10133, Italy.ORCID 0000-0003-3419-4205
Matteo ChinazziLaboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.
Jessica T DavisLaboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.
Kunpeng MuLaboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.
Ana Pastore Y PionttiLaboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.
Xinyue XiongLaboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.
Andrew ZhengOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.
Jackie BaekOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.
Vivek FariasSloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142.
Andreea GeorgescuOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.
Retsef LeviSloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142.
Deeksha SinhaOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.ORCID 0000-0002-9788-728X
Joshua WildeOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.
Georgia PerakisMassachusetts Institute of Technology, Cambridge, MA 02142.ORCID 0000-0002-0888-9030
Mohammed Amine BennounaMassachusetts Institute of Technology, Cambridge, MA 02142.ORCID 0000-0002-9123-8588
David Nze-NdongMassachusetts Institute of Technology, Cambridge, MA 02142.
Divya SinghviTechnology, Operations and Statistics (TOPS) group, Stern School of Business, New York University, New York, NY 10012.
Ioannis SpantidakisMassachusetts Institute of Technology, Cambridge, MA 02142.ORCID 0000-0002-5149-6320
Leann ThayaparanMassachusetts Institute of Technology, Cambridge, MA 02142.
Asterios TsiourvasMassachusetts Institute of Technology, Cambridge, MA 02142.ORCID 0000-0002-2979-6300
Arnab SarkerInstitute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139.ORCID 0000-0003-1680-9421
Ali JadbabaieInstitute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139.ORCID 0000-0003-1122-3069
Devavrat ShahInstitute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139.ORCID 0000-0003-0737-3259
Nicolas Della PennaLaboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA 02139.
Leo A CeliLaboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA 02139.ORCID 0000-0001-6712-6626
Saketh SundarRiver Hill High School, Clarksville, MD 21029.
Russ WolfingerJMP Life Sciences, SAS Institute Inc, Cary, NC 27513.
Dave OsthusStatistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM 87545.ORCID 0000-0002-4681-091X
Lauren CastroInformation Systems and Modeling Group, Los Alamos National Laboratory, Los Alamos, NM 87545.
Geoffrey FairchildInformation Systems and Modeling Group, Los Alamos National Laboratory, Los Alamos, NM 87545.ORCID 0000-0001-5500-8120
Isaac MichaudStatistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM 87545.
Dean KarlenDepartment of Physics and Astronomy, University of Victoria, Victoria, BC, V8W 2Y2, Canada.
Matt KinseyJohns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Luke C MullanyJohns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.ORCID 0000-0003-4668-9803
Kaitlin Rainwater-LovettJohns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.ORCID 0000-0002-8707-7339
Lauren ShinJohns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Katharine TallaksenJohns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Shelby WilsonJohns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.
Elizabeth C LeeDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.ORCID 0000-0002-4156-9637
Juan DentDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.ORCID 0000-0003-3154-0731
Kyra H GrantzDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Alison L HillInstitute for Computational Medicine, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21218.ORCID 0000-0002-6583-3623
Joshua KaminskyDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Kathryn KaminskyUnaffiliated, Baltimore, MD 21205.
Lindsay T KeeganDivision of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT 84108.ORCID 0000-0002-8526-3007
Stephen A LauerDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Joseph C LemaitreLaboratory of Ecohydrology, School of Architecture, Civil and Environmental Engineering, École Polytechnique Fédérale de Lausanne, 1015 Lausanne, Switzerland.ORCID 0000-0002-2677-6574
Justin LesslerDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Hannah R MeredithDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Javier Perez-SaezDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Sam ShahUnaffiliated, San Francisco, CA 94122.
Claire P SmithDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.
Shaun A TrueloveDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.ORCID 0000-0003-0538-0607
Josh WillsUnaffiliated, San Francisco, CA 94122.ORCID 0000-0001-7285-9349
Maximilian MarshallDepartment of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD 21218.
Lauren GardnerDepartment of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD 21218.
Kristen NixonDepartment of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD 21218.
John C BurantUnaffiliated, Amsterdam, The Netherlands.
Lily WangDepartment of Statistics, Iowa State University, Ames, IA 50011.
Lei GaoDepartment of Finance, Iowa State University, Ames, IA 50011.ORCID 0000-0002-4707-0933
Zhiling GuDepartment of Statistics, Iowa State University, Ames, IA 50011.ORCID 0000-0002-8052-7608
Myungjin KimDepartment of Statistics, Iowa State University, Ames, IA 50011.
Xinyi LiSchool of Mathematical and Statistical Sciences, Clemson University, Clemson, SC 29634.
Guannan WangDepartment of Mathematics, College of William & Mary, Williamsburg, VA 23187.
Yueying WangDepartment of Statistics, Iowa State University, Ames, IA 50011.
Shan YuDepartment of Statistics, University of Virginia, Charlottesville, VA 22904.ORCID 0000-0002-0271-5726
Robert C ReinerInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.
Ryan BarberInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.
Emmanuela GakidouInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.
Simon I HayInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.ORCID 0000-0002-0611-7272
Steve LimInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.
Chris MurrayInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.ORCID 0000-0002-4930-9450
David PigottInstitute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.
Heidi L GurungEmerging Technologies, IEM, Inc., Bel Air, MD 21015.
Prasith BaccamEmerging Technologies, IEM, Inc., Bel Air, MD 21015.
Steven A StageEmerging Technologies, IEM, Inc., Baton Rouge, LA 70809.ORCID 0000-0001-5361-6464
Bradley T SuchoskiEmerging Technologies, IEM, Inc., Bel Air, MD 21015.
B Aditya PrakashCollege of Computing, Georgia Institute of Technology, Atlanta, GA 30308.ORCID 0000-0002-3252-455X
Bijaya AdhikariDepartment of Computer Science, University of Iowa, Iowa City, IA 52242.
Jiaming CuiCollege of Computing, Georgia Institute of Technology, Atlanta, GA 30308.
Alexander RodríguezCollege of Computing, Georgia Institute of Technology, Atlanta, GA 30308.ORCID 0000-0002-4313-9913
Anika TabassumDepartment of Computer Science, Virginia Tech, Falls Church, VA 22043.
Jiajia XieCollege of Computing, Georgia Institute of Technology, Atlanta, GA 30308.ORCID 0000-0001-6530-2489
Pinar KeskinocakH. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.ORCID 0000-0003-2686-546X
John AsplundAdvanced Data Analytics, Metron, Inc., Reston, VA 20190.
Arden BaxterH. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.ORCID 0000-0002-6345-2229
Buse Eylul OrucH. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.ORCID 0000-0003-2431-3864
Nicoleta SerbanH. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.
Sercan O ArikGoogle Cloud, Sunnyvale, CA 94089.
Mike DusenberryGoogle Cloud, Sunnyvale, CA 94089.
Arkady EpshteynGoogle Cloud, Sunnyvale, CA 94089.
Elli KanalGoogle Cloud, Sunnyvale, CA 94089.
Long T LeGoogle Cloud, Sunnyvale, CA 94089.
Chun-Liang LiGoogle Cloud, Sunnyvale, CA 94089.
Tomas PfisterGoogle Cloud, Sunnyvale, CA 94089.
Dario SavaGoogle Cloud, Sunnyvale, CA 94089.
Rajarishi SinhaGoogle Cloud, Sunnyvale, CA 94089.ORCID 0000-0001-9157-674X
Thomas TsaiDepartment of Health Policy and Management, Harvard University, Cambridge, MA 02138.
Nate YoderGoogle Cloud, Sunnyvale, CA 94089.ORCID 0000-0003-4153-4722
Jinsung YoonGoogle Cloud, Sunnyvale, CA 94089.
Leyou ZhangGoogle Cloud, Sunnyvale, CA 94089.ORCID 0000-0002-2454-0082
Sam AbbottCentre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.
Nikos I BosseCentre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.
Sebastian FunkCentre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.ORCID 0000-0002-2842-3406
Joel HellewellCentre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.
Sophie R MeakinCentre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.ORCID 0000-0002-6385-2652
Katharine SherrattCentre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.ORCID 0000-0003-2049-3423
Mingyuan ZhouMcCombs School of Business, The University of Texas at Austin, Austin, TX 78712.
Rahi KalantariDepartment of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712.
Teresa K YamanaDepartment of Environmental Health Sciences, Columbia University, New York, NY 10032.ORCID 0000-0001-8349-3151
Sen PeiDepartment of Environmental Health Sciences, Columbia University, New York, NY 10032.ORCID 0000-0002-7072-2995
Jeffrey ShamanDepartment of Environmental Health Sciences, Columbia University, New York, NY 10032.ORCID 0000-0002-7216-7809
Michael L LiOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.ORCID 0000-0002-2456-4834
Dimitris BertsimasSloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142.ORCID 0000-0002-1985-1003
Omar Skali LamiOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.ORCID 0000-0002-8208-3035
Saksham SoniOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.ORCID 0000-0002-8898-5726
Hamza Tazi BouardiOperations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.ORCID 0000-0002-7871-325X
Turgay AyerH. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.
Madeline AdeeRadiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.
Jagpreet ChhatwalRadiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.
Ozden O DalgicHealth Economic Modeling, Value Analytics Labs, 34776 İstanbul, Turkey.
Mary A LaddRadiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.
Benjamin P LinasDepartment of Medicine, Section of Infectious Diseases, Boston University School of Medicine, Boston, MA 02118.
Peter MuellerRadiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.
Jade XiaoH. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.
Yuanjia WangDepartment of Biostatistics, Columbia University, New York, NY 10032.ORCID 0000-0002-1510-3315
Qinxia WangDepartment of Biostatistics, Columbia University, New York, NY 10032.
Shanghong XieDepartment of Biostatistics, Columbia University, New York, NY 10032.
Donglin ZengDepartment of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599.
Alden GreenDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Jacob BienMarshall School of Business, Department of Data Sciences and Operations (DSO), University of Southern California, Los Angeles, CA 90089.
Logan BrooksDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Addison J HuDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Maria JahjaDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Daniel McDonaldDepartment of Statistics, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada.ORCID 0000-0002-0443-4282
Balasubramanian NarasimhanDepartment of Biomedical Data Sciences, Stanford University, Stanford, CA 94305.
Collin PolitschMachine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213.ORCID 0000-0003-3727-9167
Samyak RajanalaDepartment of Statistics, Stanford University, Stanford, CA 94305.ORCID 0000-0002-5791-3789
Aaron RumackMachine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213.ORCID 0000-0002-9181-1794
Noah SimonDepartment of Biostatistics, University of Washington, Seattle, WA 98195.
Ryan J TibshiraniDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.ORCID 0000-0002-2158-8304
Rob TibshiraniDepartment of Statistics, Stanford University, Stanford, CA 94305.
Valerie VenturaDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Larry WassermanDepartment of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.
Eamon B O'DeaOdum School of Ecology, University of Georgia, Athens, GA 30602.
John M DrakeOdum School of Ecology, University of Georgia, Athens, GA 30602.ORCID 0000-0003-4646-1235
Robert PaganoUnaffiliated, Tucson, AZ 85704.
Quoc T TranCatalog Data Science, Walmart Inc., Sunnyvale, CA 94085.
Lam Si Tung HoDepartment of Mathematics and Statistics, Dalhousie University, Halifax, NS, B3H 4R2, Canada.ORCID 0000-0002-0453-8444
Huong HuynhVirtual Power System Inc, Milpitas, CA 95035.
Jo W WalkerCOVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.
Rachel B SlaytonCOVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.ORCID 0000-0003-4699-8040
Michael A JohanssonCOVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.ORCID 0000-0002-5090-7722
Matthew BiggerstaffCOVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.ORCID 0000-0001-5108-8311
Nicholas G ReichDepartment of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.ORCID 0000-0003-3503-9899

Funding

Research Services CoreP2CHD050924 · NICHD · UNIV OF NORTH CAROLINA CHAPEL HILL · PI KAREN B GUZZO · 2015 to 2026
$9.6M
Influenza Forecasting Center of Excellence at University of Massachusetts AmherstU01IP001122 · IP · UNIVERSITY OF MASSACHUSETTS AMHERST · PI REICH, NICHOLAS G · 2019 to 2023
$5.4M
Statistical methods for real-time forecasts of infectious disease: expanding dynamic time-series and machine learning approaches for pandemic scenariosR35GM119582 · NIGMS · UNIVERSITY OF MASSACHUSETTS AMHERST · PI Nicholas G Reich · 2016 to 2026
$4.8M
Statistical Methods for Integrating Mixed-type Biomarkers and Phenotypes in Neurodegenerative Disease ModelingR01NS073671 · NINDS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI WANG, YUANJIA · 2011 to 2025
$3.9M
Critical Care Informatics: Ethical considerations around the use and sharing of health-related dataR01EB017205 · NIBIB · MASSACHUSETTS INSTITUTE OF TECHNOLOGY · PI CELI, LEO ANTHONY G, MARK, ROGER GREENWOOD · 2014 to 2021
$3.8M
RFA-IP-20-003, FluMod - Center for the Multiscale Modeling of Pandemic and seasonal Flu Prevention and ControlU01IP001137 · IP · NORTHEASTERN UNIVERSITY · PI VESPIGNANI, ALESSANDRO · 2020 to 2024
$3.4M
Statistical and Machine Learning Methods to Improve Dynamic Treatment Regimens Estimation Using Real World Data.R01GM124104 · NIGMS · UNIV OF NORTH CAROLINA CHAPEL HILL · PI Yuanjia Wang, Donglin Zeng · 2018 to 2026
$3.1M
Semiparametric Analysis of Big Censored DataR01HL149683 · NHLBI · UNIV OF NORTH CAROLINA CHAPEL HILL · PI LIN, DANYU · 2020 to 2023
$1.9M
Merging machine learning and mechanistic models to improve prediction and inference in emerging epidemicsR01GM140564 · NIGMS · UNIV OF NORTH CAROLINA CHAPEL HILL · PI EDWARDS, JESSIE, LESSLER, JUSTIN · 2021 to 2024
$1.8M
Medical Research Council MC_PC_19012Medical Research Council MR/J008761/1Medical Research Council MR/R015600/1NCEZID CDC HHS U01 CK000531NCEZID CDC HHS U01 CK000538NCIRD CDC HHS U01 IP001122NCIRD CDC HHS U01 IP001137NHLBI NIH HHS R01 HL149683NIBIB NIH HHS R01 EB017205NICHD NIH HHS P2C HD050924NIGMS NIH HHS R01 GM124104NIGMS NIH HHS R01 GM140564NIGMS NIH HHS R35 GM119582NINDS NIH HHS R01 NS073671Wellcome Trust 200861/Z/16/ZWellcome Trust 210758/Z/18/Z
6 · The paper itself

Abstract

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.

Indexed as

COVID-19Data AccuracyForecastingHumansPandemicsProbabilityPublic HealthUnited StatesCOVID-19ensemble forecastforecastingmodel evaluation

Identifiers

PMID35394862
PMCPMC9169655

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.