This did not look guaranteed to work, says Matthew Chantry, machine-learning coordinator at the ECWMF, who is spending this storm season evaluating their performance. The algorithms underpinning ChatGPT were trained with trillions of words, largely scraped from the internet, but there’s no sample so comprehensive for Earth’s atmosphere. Hurricanes in particular make up a tiny fraction of the available training data. That the predicted storm tracks for Lee and others have been so good means that the algorithms picked up some fundamentals of atmospheric physics.
That process comes with drawbacks. Because machine-learning algorithms latch onto the most common patterns, they tend to downplay the intensity of outliers like extreme heat waves or tropical storms, Chantry says. And there are gaps in what these models can predict. They aren’t designed to estimate rainfall, for example, which unfolds at a finer resolution than the global weather data used to train them.
Shakir Mohamed, a research director at DeepMind, says that rain and extreme events—the weather events people are arguably most interested in—represent the “most challenging cases,” for AI weather models. There are other methods of predicting precipitation, including a localized radar-based approach developed by DeepMind known as NowCasting, but integrating the two is challenging. More fine-grained data, expected in the next version of the ECMWF data set used to train forecasting models, may help AI models start predicting rain. Researchers are also exploring how to tweak the models to be more willing to predict out-of-the-ordinary events.
Error Checks
One comparison that AI models win hands down is efficiency. Meteorologists and disaster management officials increasingly want what are known as probabilistic forecasts of events like hurricanes—a rundown of a range of possible scenarios and how likely they are to occur. So forecasters produce ensemble models that plot different outcomes. In the case of tropical systems they’re known as spaghetti models, because they show skeins of multiple possible storm tracks. But calculating each additional noodle can take hours.
AI models, by contrast, can produce multiple projections in minutes. “If you have a model that’s already trained, our FourCastNet model runs in 40 seconds on a junky old graphics card,” says DeMaria. “So you could do like a whole gigantic ensemble that would not be feasible with physically based models.”
Unfortunately, true ensemble forecasts lay out two forms of uncertainty: both in the initial weather observations and in the model itself. AI systems can’t do the latter. This weakness springs from the “black box” problem common to many machine-learning systems. When you’re trying to predict the weather, knowing how much to doubt your model is crucial. Lingxi Xie, a senior AI researcher at Huawei, says adding explanations to AI forecasts is the number one request from meteorologists. “We cannot provide a satisfying answer,” he says.
Despite those limitations, Xie and others are hopeful AI models can make accurate forecasts more widely available. But the prospect of putting AI-powered meteorology in the hands of anyone is still a ways off, he says. It takes good weather observations to make predictions of any kind—from satellites, buoys, planes, sensors—funneled through the likes of NOAA and the ECMWF, which process the data into machine-readable data sets. AI researchers, startups, and nations with limited data-gathering capacity are hungry to see what they can do with that raw data, but sensitivities abound, including intellectual property and national security.
Those large forecasting centers are expected to continue testing the models before the “experimental” labels are removed. Meteorologists are inherently conservative, DeMaria says, given the lives and property on the line, and physics-based models aren’t about to disappear. But he thinks that improvements mean it could only be another hurricane season or two before AI is playing some kind of role in official forecasts. “They certainly see the potential,” he says.