
Modern Weather Forecasts Are Stunningly Accurate - scott_s
https://www.theatlantic.com/science/archive/2019/01/polar-vortex-weather-forecasting-good-now/581605/
======
brootstrap
Just a little knowledge to drop, as a bachelor degree meteorologist and also
coder / HN'er.

Dont use your built in phone app for forecasts. Dont expect hour by hour
precision (maybe for the next few hours sure, but not 2,3,4,5 days out). Trust
nobody expect the trained professionals at NWS. These people look at ALL the
models, they watch things like a hawk. I work with some of these nerds and
it's crazy. They are like 'oh jimbob did you see the latest GFS kicking out
the low up into canada? oh yeah doug that is crazy, but the euro says blah
blah and NAM says blahblah... Their job is to literally make a forecast, for
the same area, every day. Some people do it for years and years! Think of the
experience, wisdom, little tricks and tips etc. Just go to weather.gov, look
at it for 2 minutes. Read the foreacst discussion if you want to get your
details.

~~~
pxlpshr
I find Dark Sky to be the best forecasting tool for weather within the hour.
It's almost always ahead of all the other apps in terms of the latest
forecast.

~~~
wenc
I bought the app because of its hyperlocal forecasting, but it's been known to
predict precipitation where there's none, and vice versa.

I wonder if it does better in some geographies than others (e.g. high pressure
system areas where things are more stable)

~~~
brootstrap
hyperlocal is a joke, sorry to say it. Just because you can downscale your
grid to a very small number doesnt mean good data. It just means you can do
statistics and mathy stuff to interpolate. Everyone forecasts better in stable
times such as a strong high pressure ridge. The tricky part is the edge areas
between air masses, the transition zones. This is where all the juicy weather
actually happens and it is the hardest to model

~~~
Groxx
Totally agreed on it being a marketing term, not a real prediction improvement
- it's essentially the same as any other app in terms of "will it rain" / etc.

But they do a _much_ better job of notifying about changes based on where I
physically am, instead of for the city / region as a whole. Other weather apps
I've used don't differentiate between "rain falling now 50 miles away" and
"rain falling on my head", even though the radar map clearly gives them that
info at a much finer level of detail.

------
jacques_chester
Many folks, when hearing forecasts have improved, will disagree. They can
think of many times when the forecast was wrong, perhaps hilariously or
disruptively so.

This is textbook availability bias. You overestimate the failure rate because
the failures were memorable, especially the major failures. But the far more
frequent accurate predictions are not recalled, because they don't leave as
much of an emotional impression. For the same reason folks regularly
overestimate crime rates, deaths from terrorism or rare diseases.

Meteorology has an _entire subfield_ devoted to studying the quality of
forecasts[0]. Before you bandy about numbers and anecdotes, stop and tell me
what measure you are using. Is it percentage of degrees centigrade? Well you
can't, degrees centigrade is an interval measure, not a ratio measure. If you
use Kelvins, which has a mathematically meaningful zero, the percentage
accuracy is suddenly very good. How do you count near misses of very intense
weather? Direct hits by systems that were less intense than expected? How do
you account for systems with very gradual gradients over very wide areas? How
do you account for being early by an hour or late by an hour, but nailing the
storm surge? How do you score for confidence? What's your rule for weighting
false alarms? How important is mean error vs absolute error vs variance, and
why?

Solve these and dozens of other problems of describing "accuracy" and weather
forecasters might take you seriously. But until then it's probably worth
accepting that they are the most effective profession of their kind and that
we have a lot we can learn from them.

[0]
[http://www.cawcr.gov.au/projects/verification/](http://www.cawcr.gov.au/projects/verification/)

~~~
grawprog
I worked outside every day for 4 years at least. Weather forecasts are so
unreliable I stopped listening. I had better luck trying to figure out if i'd
need rain gear in a day by just waking up and looking out the window in the
morning than looking up or listening to a weather forecast.

You really start to notice how wrong they are when you spend every day
outside. I don't notice any more working inside again. But when you rely on
weather forecasts everyday to figure out how miserable you're going to be in a
day...you really notice..they might as well just be making things up half the
time.

~~~
brootstrap
Hmmm. If you are needing down to the minute accurate timing of when rain is
going to start & stop, sure nobody can predict that, for every location in the
USA, all the time. I have to question where you are getting your weather
forecast information??? I am a bachelor degree meteorologist, I dont use any
apps that try to be 'hyper local' (omg i hate that word, it is the weather
version of AI/ML hype). Just go to the government site www.weather.gov

We can't predict exactly when it will rain, but we can get daily trends pretty
good for 3-5 days out. This means you are five days ahead of the weather. You
cant know "it will rain from 5:00am to 9:30am on february 4th, 2019". But you
can know if we have a synoptic scale system that is moving through your area
on that day, and know if there is a probability of rain.

Where do you work everyday outside? i would be happy to give you a 5 - 10 day
forecast right now actually and can tell you it will be good. I am supposed to
be writing some tests and documentation but i can drop some WX knowledge if
you need.

~~~
mannykannot
I get the feeling that this "hyper-local" mentality contributed to NYC and
surrounding areas being caught flat-footed by the first-of-season snowfall at
the end of last year. I have little understanding of meteorology, but it was
clear, just from the forecast maps, that a small change in the location and
timing of the rain/snow line would make a big difference to the outcome, and
that something more than was forecast was a distinct possibility.

As with many things, having a little understanding of the big picture can help
you get more out of weather forecasts.

~~~
brootstrap
Your last statement is the so true. An amateur forecaster can spend ten
minutes looking at the some big picture (synoptic scale) weather and know
regional trends (this part of country is dry, these guys are wet, humid etc).
This would be a pressure & wind map, showing location of high/lows, maybe some
850mb moisture charts too... But you are painting broad strokes. All of these
dumbos on HN want to know exactly when the rain is going to hit your exact
location, or you have to be within 3 degrees C modeling the surface
temperature of my street so i can either ride my bike or not. We just cant do
that with the current science & computational landscape!

After you have the synoptic scale picture, then you zoom in on your target
area and get specific if you want to. This local expertise is where you NWS
office comes into play. You want the local forecast from these guys!

~~~
labster
It's not that, it's selective memory. Everyone remembers the blown forecasts.
Even I do. Three or four years back NWS predicted a big Christmas storm in
SoCal, but the damn cut-off didn't come on shore, and the resulting offshore
flow made the day nice and warm.

And, Americans only: if you forecast temperatures of 99°F and the observed
temperature is 97°, good job. But if the observed temperature is 100°, then
you made a huge mistake. People need to know when you hit the century mark!

------
littleredstar
As an amateur astronomer/astrophotographer for the past 25+ years I've relied
on long to short term forecasts. Will it be clear for an event I want to see?
I can certainly agree the models have improved and are pretty good. Much to my
dislike on occasion such as the recent lunar eclipse. I was clouded out, that
looked likely days ahead of time.

One has to appreciate the complexity of modeling such a large and dynamic
system that the atmosphere is! There are cases where forecasting is more
difficult. Certainly getting the temperature correct within a degree or two is
one case.

Another issue is geography. I'm in middle of of USA and there is significant
amount of data collected over the continental US as systems approach me. For
those on the west coast I can see where forecasting is more challenging since
there is far less data available to input into models (current atmosphere
state 100s of kilometers out over the ocean). I'd suspect some European
countries experience same situation, those further inland benefit from
increased modelling data.

Reading the comments here I'm a bit surprised at how many quibble over slight
details. The improvements over the past decades in forecasting models and the
supercomputers that crunch the data has been significant. In most cases, like
recent cold temperatures and snow fall amounts, models converge on a good
solution as the event nears. This is consistent with idea that better data
input yields better data out.

~~~
brootstrap
The system is complex beyond our comprehension. If we really wanted to model
weather and climate better we need to know more things. Things like how much
cow farts are there? I would be happy to build a system to ingest cow fart obs
and feed them into the numerical models. FPS farts per sec yo

------
nabla9
> Meteorologists are increasingly uniting weather models and climate models,
> allowing them to project the general contours of a season as it begins.

Weather and climate models are gradually converging and both are getting
incredibly good.

They work even in astronomy. You take a climate model, set the parameters for
Mars or some exoplanets and what you get is relatively good Mars climate model
or good principled guess of of what the climate in tidal locked planet around
ultra-cool red dwarf star is (TRAPPIST-1).

------
saagarjha
> In 2009, a back-of-the-envelope study estimated that U.S. adults check the
> weather forecast about 300 billion times per year.

That's every single adult in the United States checking the weather _four
times a day_. A bit more than I would have expected…

~~~
YokoZar
This number probably includes things like swiping to the leftmost home screen
on your phone to see the weather widget at the top. When it's that easy you
can often check the weather unintentionally.

~~~
ZoomStop
That's a 2009 study, so more like every time you open your flip phone.

------
msravi
Anyone who is a weather buff should try meteogram
([https://play.google.com/store/apps/details?id=com.cloud3squa...](https://play.google.com/store/apps/details?id=com.cloud3squared.meteogram)).
You can plot a huge host of parameters (temperature, dew point, pressure,
humidity, precipitation, wind speed/direction, sun/moon azimuth/elevation,
etc.) over time, and the insights are very revealing. You can also choose
multiple data sources, and it even has options to display METAR data.

------
chewz
Just follow few models for a couple of weeks and you will notice how
inaccurate the weather forecast is.*

I do follow ECMWF and ICON and a bit of OpenWeather forecast and the
differences in forecasted temperatures are often 2-3 degrees C. And everything
with a horizon over 72 hours is just numerology. (OpenWeather showed -19C in
early days of Feb just 2 days ago, today it is -2C).

Now a model is just a model (I had some experience with industrial and
financial models) so they are good at this and not so good at that. And they
are recalculated and self-adjusted over time. I can understand.

But from the user perspective there is huge difference between +1C tomorrow
and -3C.

Windy app is using 4 models and has nice graphical comparision of forecasts.

* - What I mean here is that meteorologists have good understanding of processes but the numbers that we are getting could be as well random.

~~~
caf
_But from the user perspective there is huge difference between +1C tomorrow
and -3C._

Really? I'd read those both as "near freezing, going to need a decent coat".

~~~
cromulent
+1 means that the packed snow is going to melt on top and then probably re-
freeze, making everything super slippery. So I need to remove as much snow
from the paths as possible and spread sand, and remind the kids to be careful.

-3 means relax.

~~~
Svip
The temperatures given are usually air temperatures. If the ground is warm
enough -- which it is usually is in the cities -- then it doesn't really
matter, the snow will melt in both cases.

~~~
cromulent
Not here mate, not in one day that's for sure.

------
OkGoDoIt
I’m sure it’s easier to predict weather in some places versus others, but in
San Francisco I find the weather forecast to be wrong more often than not. I’m
often told it’s currently sunny or raining when it’s the opposite. And forget
about accurate high temperature predictions for tomorrow. At this point I
don’t know why I even bother checking.

------
mh8h
If you're looking for a graph that shows more than a projected minimum/maximum
for every day, take a look at Foreca's 15-day forecast page [0]. It shows
percentile lines for the forecasted minimum and maximum, which give a much
better idea of what to expect. It also forecasts global vs. local
precipitation. Unfortunately it is only available in Finnish.

[0]
[https://www.foreca.fi/Canada/Vancouver/15vrk](https://www.foreca.fi/Canada/Vancouver/15vrk)

Disclaimer: I used to worked for MSN Weather, where we used Foreca's feed as
the source for our weather data.

------
ggm
Back in the seventies and eighties there were four sources of impetus for
improvement in highly parallel supercomputing. Military/Nuclear, high energy
physics, oil and gas exploration (seismic analysis) and weather.

The weather people were the nicest. They could talk to almost anyone about
their work!

I remember somebody in csiro talking to me about models getting accurate at
the 1km square granular scale for a day. I think we're well beyond that for
both cell size and duration now.

Everyone loves to talk about the weather and Bob Dylan was wrong about not
needing a weatherman to know which way the wind blows

------
gruez
I skimmed this article and the linked sciencemag article and they seem to talk
about wide scale weather events (eg. hurricanes), rather than the typical
things that people look for in forecasts (yes/no rain, min/max temperature). A
freakonomics article from 2008 found that at least for precipitation
forecasts, weather forecasts don't fare much better than a simple
probabilistic model. Has forecasts like these improved in accuracy?

------
nindalf
This is an interesting example of an article that talks about gradual
improvements in our lives that we take for granted at this point. It's strange
because most articles are about dramatic deterioration. For example, it's
unlikely we will read articles about the increase in % children who are
vaccinated against basic diseases. It was 22% in 1980, 88% in 2016. Far more
likely is reading reams of text devoted to the (for now) niche anti-vaxxer
movement.

~~~
lotsofpulp
I don’t think using outbreaks of a preventable disease due to stupid people
thinking they know more than they do and causing harm to society as a whole is
overly dramatic. If anything, there should be more coverage about it so we can
get laws passed to deny non medical exemptions.

~~~
nindalf
What's the impact in terms of affected people. Hundreds of millions of
children being vaccinated, millions of them not dying young, dramatically
changing the dynamics of societies where women can now choose not to have
multiple children.... or a few hundred people getting measles. I'm not saying
the anti-vax thing isn't a problem. I'm saying that the improvements and
advancements have a higher impact on society than what a few idiots choose to
do. And yet we read much more about the latter.

~~~
lotsofpulp
The issue of people not vaccinating is specifically one which can spiral out
of control very easily, as the tools we have to fight the disease don't work
once herd immunity is lost, hence sounding the alarm bells is warranted.

Actually, now that I think about it, perhaps if the history of vaccines and
the success had been more widely disseminated, then we may not have had this
anti vaccine problem. I agree that it would be nice to read more about
successes.

------
stunningly
The actual research this is based on is: "The Quiet Revolution of Numerical
Weather Prediction" Bauer, Thorpe, Brunet, 2005.

"Stunningly Accurate" means that 7-day forecasts are now at the lower boundary
of being considered "useful" so the bar is not being set very high here.

Still not a lot to quell the skepticism that some reasonable people have about
the ability of scientists to accurately predict weather decades in the future.

~~~
organsnyder
Weather ≠ climate.

~~~
nonsequitter
right. climate is a more complex system. more variables. more unknowns.
exponentially larger prediction area. unknown base states. so it should be
much easier to predict right?

~~~
Majromax
> so it should be much easier to predict right?

Yes, because the goals of the forecast are different. A weather forecast seeks
to predict a future position within the phase space of the system; a climate
forecast seeks to predict the overall shape of that phase space.

"It will be 1C warmer in February on average" is a useless prediction if I'm
deciding whether or not to wear a heavy coat tomorrow, since day-by-day
variability swamps that average. But it is a very useful prediction if I'm
designing infrastructure that needs to last 50 years.

------
bsbechtel
I only see forecast errors for hurricanes listed as the cited data. Maybe I
missed the other cited sources for data measuring forecast accuracy, but
hurricane forecast accuracy, which is a storm that covers hundreds of square
kilometers, doesn't seem like enough data to justify making the general claim
'modern weather forecasts' are stunningly accurate.

------
ljcn
Shout out to Climendo.

I use the android app and it gives me a simple comparison of 4-5 weather
forecasts, an "average" forecast, and a "certainty" (a measure of consensus).

Obviously the average of the ensemble averages isn't as good as having the
underlying ensembles themselves, but it's nice to see days when the forecasts
agree and days when they have no idea!

------
galfarragem
Best forecasts that I know (accurate enough to people come and ask me):
[https://www.windguru.cz/](https://www.windguru.cz/)

Notes:

\- Only the GFS27KM is reliable on free version and within next 48-72h.

\- Temperatures are always conservative on extremes. When it says 37C, expect
40C. When 2C expect 0C.

------
T3OU-736
Intriguing. Barely a mention (buried and oblique) that there are different
weather models, and almost no discussion about relative accuracy or the
reasons for the difference. Especially contrasting with the last paragraph
being explicitly "Rah rah America" with the fact that the US NWS' weather
models are less accurate than the European ones (hurricane Sandy was a huge
wake-up call)

~~~
brootstrap
They also dont even reference any USA models in the article. The GFS was also
showing this big cold snap, not just the euro!

------
libso
What's missing in most of the weather services is the recent history forecast
vs observation.

------
moocowtruck
why is my weatherman always wrong? :(

~~~
Niksko
Because we tend to remember the few times where they get it very wrong (which
is always going to happen because of the chaotic nature of weather) and ignore
the large percentage of the time when they're right.

------
voyager2

      I use the forecast put out by weather.gov that's supposedly tailored

to the square mile because it gets the 12 hour forecast right about 40% of the
time. The others are worse.

    
    
      Of course, when they say "chance of precipitation is 80%, less than an
      inch possible" and it doesn't rain, the forecast is semantically
      "correct."
    
      Like the El Nino impact on the SE US, where they forecast a 50% chance

of drier colder weather and a 50% chance of warmer wetter weather, it's nearly
impossible to be wrong.

    
    
      Maybe this is the sort of obfuscatory probabilistic forecast Mr. Meyer

is counting as "accurate."

~~~
maaaats
> _it gets the 12 hour forecast right about 40% of the time_

Where are these figures from? Your emotions, or something scientifically
rigid? If the former, having this discussion is meaningless.

~~~
voyager2
"chance of precipitation is 80%, less than an inch possible"

How would you determine, in a scientifically rigid manner, the limits on
conditions which would validate that forecast as "right?" Or the inverse. What
conditions would invalidate it as "not right?"

~~~
scott_s
In a top-level comment upthread, jacques_chester posted a link to an overview
of forecast verification methods:
[http://www.cawcr.gov.au/projects/verification/](http://www.cawcr.gov.au/projects/verification/)

------
sleepysysadmin
The recent snow storm in my area, 4 days in advance they were predicting 30cm
of snow. 3.5 days in advance were 25cm of now. 3 days in advanced 15cm of
snow. 2 days in advanced 5-10cm of snow. 1 day in advance 15cm of snow. We got
about 10cm of snow.

Sorry but that's stunningly inaccurate.

~~~
simonh
4 days in advance they correctly predicted you would get snow and that it
would be on the order of tens of centimeters thick. That's really good. Over
the intervening days they revised their estimate down and the snowfall was
eventually lower than the initial estimate so the accuracy improved. Their
final estimate was again correct that you did get snow, which is the key
prediction, and they even got the depth at your precise location right to
within 33%. That's fantastic, and far better than was possible for the vast
majority of my lifetime.

Also, the fact that their predictions were all rounded to the nearest 5cm
should give you a clue at the expected accuracy and granularity of the final
prediction. Missing the target by only one 'unit' of measure is pretty decent.

~~~
scott_s
There's also the fact that predicting amount of snow is particularly difficult
because the same amount of water will yield vastly different snowfalls with
single-degree deviations of the temperature.

------
ancorevard
No, both our weather forecasting and climate model forecasting are still very
very weak and poorly understood. Predicting the future is still hard for us
humans.

~~~
amdavidson
Sources?

