Preprint finds probabilistic AI models outperform ECMWF ensemble on record-breaking heat extremes

A new arXiv preprint says probabilistic AI weather models outperformed the European Centre for Medium-Range Weather Forecasts’ flagship forecasting ensemble not just overall, but specifically on record-breaking heat extremes — a direct challenge to earlier studies that concluded physics-based systems did better on the most severe events.

That matters because extreme heat is among the deadliest weather hazards, and better forecasts can shape public health alerts, power-grid planning, insurance decisions and emergency response. The paper’s broader claim is also consequential: whether AI looks weak or strong on rare extremes may depend heavily on how it is judged. The authors argue that earlier comparisons often treated probabilistic AI systems as if they were single-value forecasts, an approach they say unfairly penalizes models built to estimate a range of possible outcomes.

The study, posted on arXiv on Oct. 14, 2026, is a preprint and has not been peer-reviewed. It is titled “Probabilistic AI models outperform numerical weather prediction models on record breaking heat extremes” and was written by Jakob Benjamin Wessel, Fiona Spuler, Christopher A. T. Ferro, Marlene Kretschmer and Sebastian Sippel of Leipzig University and the University of Exeter. The researchers compared two operational probabilistic AI weather models — WeatherNext2 and WeatherGen, also referred to as GenCast — against the ECMWF Integrated Forecasting System ensemble, one of the world’s leading physics-based operational weather forecasting systems. They tested forecasts for 2-meter air temperature over global land at 0.25-degree resolution during calendar year 2022, with lead times from 12 to 120 hours in 12-hour steps.

The headline numbers were strongest at 72 hours. WeatherNext2 improved the study’s main probabilistic accuracy measure, known as the continuous ranked probability score, or CRPS, by 25% relative to the IFS ensemble. WeatherGen improved it by 17%. Post-processing the IFS forecasts with a statistical correction known as EMOS improved IFS by 14%, but still did not match the AI models. On record-breaking heat in gridded verification data, 39% to 45% of observed record events were hotter than every member of the IFS ensemble. For WeatherNext2, that share was 9% to 17%; for WeatherGen, 11% to 21%. In practical terms, the AI ensembles missed far fewer record heat events.

The result was not limited to gridded datasets. Using 5,778 HadISD weather stations, the authors found the AI systems also performed better on station-observed records. For record events at stations, WeatherNext2 improved the Brier score — a standard measure of probabilistic forecast quality — by 10% at 72 hours and 19% at 84 hours compared with IFS. WeatherGen improved it by 7% at 72 hours and 16% at 84 hours. The AI models also missed fewer station records: about 50% to 52% of station record events were above all AI ensemble members, versus 62% to 63% for IFS. The authors also reported that the AI models were better calibrated in the extreme tail, meaning their predicted probabilities for very rare heat events lined up more closely with what was observed.

That finding cuts against a 2026 Science Advances study by Zhang and colleagues, which reported that physics-based models outperformed AI on record-breaking extremes. The new paper says that conclusion depended in part on “unsuitable verification choices” — specifically, deterministic-style comparisons that reduce a probabilistic forecast to a single best guess. By evaluating the full forecast distributions instead, the authors argue, the ranking reverses and AI comes out ahead. As the abstract put it, the models “clearly outperform the ECMWF IFS ensemble, both raw and post-processed, including for record-breaking events.”

The paper’s limits are significant. It covers just one year, 2022, and only one variable, near-surface air temperature. The results are also sensitive to the verification data used and to the training data behind the AI systems. And while the station analysis strengthens the case, the authors say more local and station-based validation is needed. For now, the study does not settle the debate over AI and extreme-weather forecasting. But it does sharpen it, suggesting that for the hardest and most consequential heat forecasts, the choice of evaluation method may be nearly as important as the choice of model.