Oxford preprint finds GraphCast and NeuralGCM fail basic coordinate-change tests

An Oxford-led preprint says two of the best-known machine-learning weather models, GraphCast and NeuralGCM, failed simple stress tests that conventional physics-based weather models should pass. The finding, posted Tuesday on arXiv, raises questions about whether some leading AI forecast systems are learning the underlying rules of the atmosphere or relying too heavily on geography-specific patterns in historical data.

The paper, “Spatial Generalization Tests for Machine Learning-based Weather Models to Assess Physical Consistency,” was posted as a preprint on arXiv as 2607.20716v1 on July 22 by University of Oxford researchers Maren Höver, Milan Klöwer, Christian Schroeder de Witt and Hannah M. Christensen. Because it is a preprint, it is public but not described here as peer-reviewed. The authors’ central argument is that if an AI weather model cannot handle physically equivalent versions of Earth under simple coordinate changes, that is a warning sign about how well it has learned atmospheric physics rather than statistical regularities tied to today’s map.

To test that, the researchers applied three spatial generalization checks. In plain terms, they flipped the planet in latitude, flipped it in longitude, and rotated it by 180 degrees in longitude. They also changed boundary conditions and forcings — the background inputs that shape a simulation, such as land-sea layout and other external constraints — so the altered setup remained physically consistent. As the team put it in an abstract presented earlier this year at the European Geosciences Union General Assembly 2026, the tests involved “reversing the entirety of the input data and boundary conditions in latitude (Test 1), reversing them in longitude (Test 2), as well as rotating them by 180˚ in longitude (Test 3), while keeping all aspects of the simulation physically consistent.”

The reported result was stark. According to the arXiv abstract, “We reverse or rotate the planet in longitude or latitude under the model’s coordinate system and adapt all boundary conditions and forcings accordingly. Physics-based general circulation models simulate a rotated/reversed planet with only rounding errors, but GraphCast and NeuralGCM fail these tests.” The authors argue that such failures point to “unphysical variable mappings based on correlation rather than causation,” and that machine-learning climate models should be built to pass tests like these.

The claim matters because GraphCast and NeuralGCM are prominent systems in AI weather prediction. GraphCast is a DeepMind forecasting model first described in Science in 2023. NeuralGCM, published in Nature in 2024 by Google Research and collaborators, is a hybrid model that combines neural networks with physical modeling. One notable detail is that Klöwer, one of the new paper’s Oxford co-authors, was also a co-author on the earlier NeuralGCM Nature paper, meaning the critique is not coming only from outsiders to the field.

The broader issue is generalization beyond today’s climate. Weather and climate models are expected to reflect physical laws that should not depend on where continents and oceans happen to sit in a model’s coordinate system. The Oxford authors argue that failure on these tests could be a warning sign for using machine-learning systems under changing real-world conditions such as sea-ice loss, land-use change or warmer oceans. As background, NeuralGCM’s 2024 Nature paper also cautioned that the model does not extrapolate to substantially different future climates.

The work had already been presented publicly at EGU 2026 under a closely related title before its arXiv posting. As of July 24, the source material did not identify an independent replication of the findings or a public response or formal rebuttal from the teams behind GraphCast or NeuralGCM.