TY - JOUR A1 - Sherratt, Katharine A1 - Srivastava, Ajitesh A1 - Ainslie, Kylie A1 - Singh, David E. A1 - Cublier, Aymar A1 - Marinescu, Maria Cristina A1 - Carretero, Jesus A1 - Garcia, Alberto Cascajo A1 - Franco, Nicolas A1 - Willem, Lander A1 - Abrams, Steven A1 - Faes, Christel A1 - Beutels, Philippe A1 - Hens, Niel A1 - Müller, Sebastian A1 - Charlton, Billy A1 - Ewert, Ricardo A1 - Paltra, Sydney A1 - Rakow, Christian A1 - Rehmann, Jakob A1 - Conrad, Tim O.F. A1 - Schütte, Christof A1 - Nagel, Kai A1 - Abbott, Sam A1 - Grah, Rok A1 - Niehus, Rene A1 - Prasse, Bastian A1 - Sandmann, Frank A1 - Funk, Sebastian T1 - Characterising information gains and losses when collecting multiple epidemic model outputs JF - Epidemics N2 - Collaborative comparisons and combinations of epidemic models are used as policy-relevant evidence during epidemic outbreaks. In the process of collecting multiple model projections, such collaborations may gain or lose relevant information. Typically, modellers contribute a probabilistic summary at each time-step. We compared this to directly collecting simulated trajectories. We aimed to explore information on key epidemic quantities; ensemble uncertainty; and performance against data, investigating potential to continuously gain information from a single cross-sectional collection of model results. Methods We compared July 2022 projections from the European COVID-19 Scenario Modelling Hub. Five modelling teams projected incidence in Belgium, the Netherlands, and Spain. We compared projections by incidence, peaks, and cumulative totals. We created a probabilistic ensemble drawn from all trajectories, and compared to ensembles from a median across each model’s quantiles, or a linear opinion pool. We measured the predictive accuracy of individual trajectories against observations, using this in a weighted ensemble. We repeated this sequentially against increasing weeks of observed data. We evaluated these ensembles to reflect performance with varying observed data. Results. By collecting modelled trajectories, we showed policy-relevant epidemic characteristics. Trajectories contained a right-skewed distribution well represented by an ensemble of trajectories or a linear opinion pool, but not models’ quantile intervals. Ensembles weighted by performance typically retained the range of plausible incidence over time, and in some cases narrowed this by excluding some epidemic shapes. Conclusions. We observed several information gains from collecting modelled trajectories rather than quantile distributions, including potential for continuously updated information from a single model collection. The value of information gains and losses may vary with each collaborative effort’s aims, depending on the needs of projection users. Understanding the differing information potential of methods to collect model projections can support the accuracy, sustainability, and communication of collaborative infectious disease modelling efforts. Data availability All code and data available on Github: https://github.com/covid19-forecast-hub-europe/aggregation-info-loss KW - Virology KW - Infectious Diseases KW - Public Health, Environmental and Occupational Health KW - Microbiology KW - Parasitology KW - Epidemiology Y1 - 2024 U6 - https://doi.org/10.1016/j.epidem.2024.100765 SN - 1755-4365 VL - 47 PB - Elsevier BV ER -