{In Thick Yorkshire Accent} - 'Luxury! In my day, we used to call it "maths"! Two-hundred and fifty of us huddled together in a bomb shelter in Bracknell[1] poring over oceanographic maps by gaslight and trying not to sneeze into the sergeant's tea.'
You were lucky to have ones and twos! The best we could manage was to rifle through tea leaves at the bottom of a cracked cup to divine the forecast for the next six hours in Thebes. And when we would read the leaves wrong, our dad would thrash us in two with bronze bread knife and dance about our graves singing Ode to Dionysus.
It's all just marketing. The companies running these advanced chatbots want to tap into tech mystique to give the impression that LLMs are on the cusp of being the intelligent agents we see in movies and books, because just admitting they're just implementing statistical patterns isn't sexy. They need investors, and investors are throwing money at anything called AI
Eh, "AI" has always (at least a decade+) been thrown around as a catch all term for things that should have been called ML or DS or statistics (though I find that term a bit reductive, it's like saying it's all just math).
At least these days the LLMs have gotten us a little closer to the original promise of AI.
> In the first effort, the team behind Pangu-Weather, trained their system on 39 years of weather data and then asked it to make predictions based on current weather patterns. They found that it was as accurate at doing so as existing systems, and did its work in just a fraction of the time.
That's awesome -- years of analysis paying off in a very cool and interesting way. It says that this particular model doesn't predict precipitation -- just atmospheric conditions -- but there's a lot of really interesting potential, here.
Older books talked about two methods of predicting the weather: (1) based on atmospheric modelling, and (2) based on historical trends. The American Boy's Handy Book
It's specifically referring to these forecast models having accuracy about on par with the state-of-the-science global numerical weather forecast models. "Accuracy" here specifically means esoteric metrics like the "500mb anomaly correlation coefficient" (basically a summary statistic that tells you how well the 3D atmosphere fields predicted by the model match what we observe later on).
This entire class of global numerical weather forecast models has had more-or-less a monotonic increase in forecast accuracy over the past five decades. E.g., a 72 hour forecast from the current generation of these models has similar error statistics to a 24 hour forecast from its predecessors in the early 2000's.
What's special about these AI forecast models is that they are significantly cheaper to run than the existing global numerical weather forecast models. Modern meteorology involves a great deal of statistical analysis to overcome chaotic uncertainty. One way we build these statistical analyses is to run dozens of forecasts with the same model, using slightly different initial conditions, to see how the forecasts diverge. But these ensembles are very under-disperse - a few dozen members just doesn't fully sample the uncertainty. Now, if you can run 1000x the number of ensemble members, a whole new world of possibility opens up.
And that's before you consider just training the AI system to directly output a posterior distribution representing this uncertainty in the first place...
Thanks for the explanation but my question was not about the cost but about the accuracy. Or is it the case that because of the reduced cost there is an opportunity to improve accuracy because more scenarios can be executed?
The problem in this world of AI/weather is that "accuracy" is an extremely fuzzy concept. The leading pack of AI-NWP models (NVIDIA's FourCastNet, DeepMind's Graphcast, Huawei PanguWeather), when compared on an apples-to-apples basis with the leading pack of traditional NWP systems (NOAA GFS and ECMWF HRES) have similar accuracy metrics. But here, "accuracy" is an esoteric term that is far removed from how an end user would perceive how "good" any of these given models are at predicting tomorrow's afternoon high temperature at their house.
In the world of meteorology, the way you build an "accurate" (e.g. "user-perceived accuracy") forecast is to consume the entire previous class of forecasts and apply statistics/ML to post-process them. The greater your ability to probe uncertain in the forecasts from these models - e.g. by running larger ensembles or tailoring the ones you run to try to quantify the uncertainty more explicitly - the better your opportunity to improve those 'accuracy" metrics. So yes - the opportunity here is running larger sets of tailored forecast simulations as a way to statistically optimize forecast accuracy.
Was telling people about how good DarkSky was over the weekend!
It's a shame the blog got pulled. They didn't do such a thorough job though as the Vimeo videos are still there (was able to view them via the web archive link you posted... You need to login as they're unrated)
... in the console, and it's almost entirely F. (except for the text summary at the top, which can include other numbers, so isn't safe to mess with this way). You could turn this into a bookmarklet and do this one-click. https://caiorss.github.io/bookmarklet-maker/
I know, totally impractical, no use on a phone, etc. I just wondered how easy it would be to fix.
(edited to add: lol, so mcluck had the same idea. the hour-temp and regex stuff in mine is because replacing just temp__value leaves the hour-by-hour temps in the top panel, which include the degree symbol as part of the text, so I want to leave that alone)
I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. One would think this would be one of the first areas that would adopt AI given the data available.
Also that these new AI systems are performing at the same level of the old non-AI approaches I feel is a real testament to the developers of those systems. This is not something we see in many other cases.
ML/AI has been used in the weather forecasting world since the 1970's - most often to post-process forecast model output to better calibrate it against observations and correct for biases.
> One would think this would be one of the first areas that would adopt AI given the data available.
Actually, the challenge has been that there _isn't_ enough data available. Sure, we have lots of satellite observations and many other sources, but none of these paint a holistic picture of the atmosphere of the sort you'd need to actually forecast the weather with any precision. ERA-5 - a model-based "re-analysis" that assimilates many observations and tries to create a coherent picture - is only a few years old, and has been the keystone that unlocked all of this development over the past few years.
They spent billions and decades on software and models that work good enough; I think AI would work too, given enough data to learn, but would it be better or just different?
I mean keeping in mind the number of variables involved and, more recently, rapid climate change, el niño and forest fires that are much more difficult for an AI to keep in mind (I think).
It's a competitor to existing models and supercomputers, but it has to prove its reliability first.
I mean as we're seeing right here, they are just as good in their testing stage while not taking several hours to run. They're more efficient to run and calculate since they don't need to go through a lengthy process to determine the weather, just take advantage of the cyclic nature of nature to do the work for us.
The problem is that we are entering an era of rapid climate shift. Relying primarily on historical data doesn't work that well when you are trying to predict unprecedented weather events. The laws of physics, on the other hand, have not changed - so a simulation will still work just fine.
No we aren't actually. We are still well within historical data, since we use that still to calculate future weather patterns, and there hasn't been anything unprecedented that would trip up a model like this either.
No, what they are saying is events are not so far out of bounds that the models fall apart. Either there have been no black swans spotted yet, or in conjunction with other numerical models the outliers are correctly predicted.
A lot of people are using neural network models to model fields in variable resolution, solve PDEs and such. I know a bit about astrophysical modeling and you sure can't afford to model every part of a star in the same amount of detail. You can pick some special coordinate system or make the grid vary in size or you can use a neural network to model some f(x,y,z) and the network can allocate its own resources to put more detail where you need it.
PINNs are neat but by my understanding they’re conceptually not too different from traditional numerical solution methods. The solution to your PDE is some complicated function that is not available in closed-form, so you require a numerical approximation of that function. Traditional methods use various kinds of grid discretization systems (possibly with interpolation schemes). PINNs replace the grid with a parametric continuous function. I’m not sure how PINNs handle discontinuities like shocks though.
I'm actually not surprised that people think 'AI' can fix everything including weather prediction. I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry...
This aint a retail product recommendation. If the AI makes a bad choice it will cost millions/billions of dollars and we could lose many MANY lives.
> I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry...
The problem is that they offer little to no advantage over the highly optimized ML-based forecast post-processing systems widely in use in the industry. You see an awful lot of hype from start-ups proclaiming their AI forecasts are "the most accurate ever"... when in reality they barely improve at all over the status quo that can be achieved with rather simple statistical modeling.
