Artykuł w czasopiśmie
Brak miniatury
Licencja
CC-BYCC-BY - Uznanie autorstwa
 

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Uproszczony widok
dc.abstract.enBackground: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks.
dc.affiliationUniwersytet Warszawski
dc.contributor.authorZieliński, Jakub
dc.contributor.authorRakowski, Franciszek
dc.contributor.authorNowosielski, Jędrzej
dc.contributor.authorNiedzielewski, Karol
dc.contributor.authorFunk, Sebastian
dc.contributor.authorBracher, Johannes
dc.contributor.authorOzanski, Tomasz
dc.contributor.authorKrueger, Tyll
dc.contributor.authorIdzikowski, Radoslaw
dc.contributor.authorFilinski, Maciej
dc.contributor.authorBodych, Marcin
dc.contributor.authorPrzemyslaw, Biecek
dc.contributor.authorScholz, Markus
dc.contributor.authorKheifetz, Yuri
dc.contributor.authorHolger, Kirsten
dc.contributor.authorPabjan, Barbara
dc.contributor.authorKisielewski, Jan
dc.contributor.authorSzczurek, Ewa
dc.contributor.authorSemeniuk, Marcin
dc.contributor.authorRadwan, Maciej
dc.contributor.authorMoszyński, Antoni
dc.contributor.authorKrupa, Bartosz
dc.contributor.authorGruziel-Slomka, Magdalena
dc.contributor.authorGogolewski, Krzysztof
dc.contributor.authorGambin, Anna
dc.contributor.authorDreger, Filip
dc.contributor.authorBartczuk, Rafal P
dc.contributor.authorVenkatramanan, Srinivasan
dc.contributor.authorPorebski, Przemyslaw
dc.contributor.authorMarathe, Madhav
dc.contributor.authorLewis, Bryan
dc.contributor.authorHurt, Benjamin
dc.contributor.authorAdiga, Aniruddha
dc.contributor.authorMontero-Manso, Pablo
dc.contributor.authorSrivastava, Ajitesh
dc.contributor.authorFarcomeni, Alessio
dc.contributor.authorMingione, Marco
dc.contributor.authorLasinio, Giovanna Jona
dc.contributor.authorBartolucci, Francesco
dc.contributor.authorTarantino, Barbara
dc.contributor.authorGiudici, Paolo
dc.contributor.authorLovison, Gianfranco
dc.contributor.authorCatala, Marti
dc.contributor.authorDivino, Fabio
dc.contributor.authorPennoni, Fulvia
dc.contributor.authorZibert, Janez
dc.contributor.authorKuhlmann, Alexander
dc.contributor.authorZimmermann, Tom
dc.contributor.authorRodloff, Arne
dc.contributor.authorBurgard, Jan Pablo
dc.contributor.authorPrats, Clara
dc.contributor.authorLopez, Daniel
dc.contributor.authorAlvarez, Enric
dc.contributor.authorAlonso, Sergio
dc.contributor.authorMorina, David
dc.contributor.authorAznarte, Jose L
dc.contributor.authorGuzman-Merino, Miguel
dc.contributor.authorSingh, David E
dc.contributor.authorHotz, Thomas
dc.contributor.authorHeyder, Stefan
dc.contributor.authorBock, Wolfgang
dc.contributor.authorZiarelli, Giovanni
dc.contributor.authorParolini, Nicola
dc.contributor.authorArdenghi, Giovanni
dc.contributor.authorBejar, Benjamin
dc.contributor.authorRedlarski, Grzegorz
dc.contributor.authorPriesemann, Viola
dc.contributor.authorMohr, Sebastian
dc.contributor.authorDehning, Jonas
dc.contributor.authorSaksham, Soni
dc.contributor.authorLi, Michael Lingzhi
dc.contributor.authorDimitris, Bertsimas
dc.contributor.authorPribylova, Lenka
dc.contributor.authorKraus, David
dc.contributor.authorKraus, Andrea
dc.contributor.authorEclerova, Veronika
dc.contributor.authorMaruotti, Antonello
dc.contributor.authorLoro, Pierfrancesco Alaimo Di
dc.contributor.authorOsthus, Dave
dc.contributor.authorMichaud, Isaac
dc.contributor.authorFairchild, Geoffrey
dc.contributor.authorCastro, Lauren
dc.contributor.authorMeakin, Sophie R
dc.contributor.authorBosse, Nikos I
dc.contributor.authorReina, Borja
dc.contributor.authorAlvarez, Cesar Perez
dc.contributor.authorZajicek, Milan
dc.contributor.authorSmid, Martin
dc.contributor.authorTucek, Vit
dc.contributor.authorVillanueva, Inmaculada
dc.contributor.authorWalraven, Robert
dc.contributor.authorBudzinski, Jozef
dc.contributor.authorSuchoski, Bradley
dc.contributor.authorStage, Steven
dc.contributor.authorGurung, Heidi
dc.contributor.authorBaccam, Prasith
dc.contributor.authorRodiah, Isti
dc.contributor.authorLange, Berit
dc.contributor.authorFuhrmann, Jan
dc.contributor.authorWlazlo, Jaroslaw
dc.contributor.authorSchneider, Johanna
dc.contributor.authorMohring, Jan
dc.contributor.authorLeithauser, Neele
dc.contributor.authorBarbarossa, Maria Vittoria
dc.contributor.authorMeinke, Jan H
dc.contributor.authorKrymova, Ekaterina
dc.contributor.authorPottier, Loic
dc.contributor.authorThanou, Dorina
dc.contributor.authorSun, Tao
dc.contributor.authorObozinski, Guillaume
dc.contributor.authorTrnka, Jan
dc.contributor.authorWang, Lijing
dc.contributor.authorWattanachit, Nutcha
dc.contributor.authorWang, Yijin
dc.contributor.authorSheldon, Daniel
dc.contributor.authorReich, Nicholas G
dc.contributor.authorRay, Evan L
dc.contributor.authorGibson, Graham
dc.contributor.authorUllrich, Alexander
dc.contributor.authorAbbott, Sam
dc.contributor.authorWolffram, Daniel
dc.contributor.authorDeuschel, Jannik
dc.contributor.authorSandmann, Frank
dc.contributor.authorPrasse, Bastian
dc.contributor.authorNiehus, Rene
dc.contributor.authorJohnson, Helen
dc.contributor.authorGrah, Rok
dc.contributor.authorGruson, Hugo
dc.contributor.authorSherratt, Katharine
dc.date.accessioned2024-01-25T17:33:15Z
dc.date.available2024-01-25T17:33:15Z
dc.date.copyright2023-04-21
dc.date.issued2023
dc.description.accesstimeAT_PUBLICATION
dc.description.financePublikacja bezkosztowa
dc.description.versionFINAL_PUBLISHED
dc.description.volume12
dc.identifier.doi10.7554/ELIFE.81916
dc.identifier.issn2050-084X
dc.identifier.urihttps://repozytorium.uw.edu.pl//handle/item/116919
dc.identifier.weblinkhttps://cdn.elifesciences.org/articles/81916/elife-81916-v2.pdf
dc.languageeng
dc.pbn.affiliationcomputer and information sciences
dc.relation.ispartofeLife
dc.relation.pages1-19
dc.rightsCC-BY
dc.sciencecloudnosend
dc.titlePredictive performance of multi-model ensemble forecasts of COVID-19 across European nations
dc.typeJournalArticle
dspace.entity.typePublication