
Hurricane Joaquin Forecast: Why U.S. Weather Model Has Fallen Behind - jsm386
http://www.nytimes.com/2015/10/03/upshot/hurricane-joaquin-forecast-european-model-leads-pack-again.html
======
aaronbrethorst
If you live in the Pacific Northwest, the blog of the professor mentioned in
the article, Cliff Mass, is a must-read:
[http://cliffmass.blogspot.com](http://cliffmass.blogspot.com)

~~~
ZainRiz
Or if you want the abridged version check out the tldr feed:
[https://twitter.com/cliffmasstldr](https://twitter.com/cliffmasstldr)

------
chimtim
This was a fascinating read. I found the initial bit interesting that other
models were ahead because they simply had more raw computational power, though
it later adds that there may be other subtle differences.

~~~
dhimes
They indicated that the other models were using the change of the atmospheric
variables in their initial conditions. (Neumann boundary conditions?) The
models may actually be fundamentally different.

------
heimatau
Anyone in this field says it's extremely complicated. NASA (or ESA) probably
would solve this, if we didn't underfund them so much. To those advocating to
privatizing, I don't see how the private sector could solve it by themselves.
Just like McDonalds didn't land on the moon. It's extremely complicated. The
task is going to require a lot more scientists and innovators to understand
our weather patterns, it's my opinion that the best way to do that, is through
the large governments of the world (not just USA's govt).

~~~
karmelapple
Space exploration, nor the serving of food to those in space, was a goal of
McDonald's. There was no sufficient market opportunity. However, is there a
market opportunity for improved weather prediction? Who would benefit and make
money?

~~~
greglindahl
ECMWF charges for their weather forecasts, which are more accurate than the
free American ones.

There are lots of small commercial weather forecasting firms who sell to
people like securities traders. A single high-temperature day can reduce corn
yield by 1%.

Here's a startup that had a $1 billion exit to Monsanto:
[http://www.theverge.com/2013/10/10/4823004/monsanto-bets-
the...](http://www.theverge.com/2013/10/10/4823004/monsanto-bets-the-farm-on-
big-data) Their business is a combination of weather and climate.

In short, there's a big market for this stuff.

------
lacker
It seems quite strange if computing power is really the bottleneck here. What
sort of architecture would you have for a weather prediction system where you
would spend more on computing power than researchers' salaries?

~~~
lsiq
There is a very good reason computing power is the bottleneck and it is
because of how the weather has to be modeled. Weather models simulate the
conditions in the cells of a lattice that is overlaid over geographic areas.
To improve accuracy, the cells need to be shrunk to smaller sizes as they have
been for a long time. However, every time the size of the cells in the
lattices are halved you need 8 times the computing power to perform the
calculations of the 3D space. Because of Moore's Law, it takes many years for
it to make sense for weather centers to upgrade their infrastructure and see
significant improvements in their predictions.

~~~
pinewurst
Cells also dynamically influence neighboring cells in all dimensions, making
the highest performance, lowest latency interconnects essential. Interconnect
pricing is a significant chunk of these large system costs.

I occasionally hear statements wondering why weather modeling can't simply be
distributed to desktop or cloud and can't, for some strange reason, get people
to understand that connectivity is almost as important as computation itself
for this.

~~~
metaobject
We have a few clusters at work that we use for numerical weather forecasting
using the WRF weather model and some other custom applications. And, indeed,
the InfiniBand interconnect was a hefty portion of the overall cost of the
clusters. At one point there were people considering Gigabit Ethernet instead
of IB, but thankfully it was shot down. The lower latency with IB has a
greater impact on MPI-based applications than some people realize.

~~~
tcdent
So, really, no commercial "cloud" product meets these needs? So many other
industries are beyond managing hardware; seems like an obvious opportunity for
a product.

~~~
pinewurst
Azure supports RDMA over IB on certain node types but (my 2 cents) without
more detailed information & control over topology it's hard to run non-trivial
MPI jobs in anything close to an optimal way. The hiding of the "secret sauce"
that cloud vendors do may not be important for most commercial users but it's
a roadblock for a lot of HPC.

------
rch
I only have a few unrelated anecdotes to draw from, but my impression is that
there is an outsized political influence on where in the US the currently
allocated R&D funding is being spent.

------
deciplex
> _Maybe it will be Joaquin’s false alarm, not another Hurricane Sandy, that
> gets America to make the G.F.S. great again._

Or maybe it will prompt the usual suspects to "privatize" the whole thing i.e.
cut funding under the assumption that private enterprise, or churches, or
whatever the hell else, will pick up the slack.

------
jonknee
When the party that controls Washington has not believing in climate change as
a party platform I don't see a dramatic increase in budget for climate
scientists coming anytime soon. Thankfully we can use the European model!

~~~
happyscrappy
Please don't conflate climate and weather.

~~~
jonknee
The GOP has consistently wished to cut the budget of the National Weather
Service:

[http://www.slate.com/blogs/weigel/2011/03/11/gop_s_continuin...](http://www.slate.com/blogs/weigel/2011/03/11/gop_s_continuing_resolution_cuts_funding_for_national_weather_service_fema.html)

------
sandworm101
Do those in charge actually want accurate predictions? The hurricane is going
to go where it is going to go, the model has no impact on the realworld path.
If we know the path with certainty then those in the path can take appropriate
measures. But what is we enjoy those measures and want to do them regardless
of where the storm will actually hit?

What a storm might be approach the executive branch of the local government
goes into emergency mode. Emergency funds are spent on preparations. Some
government workers are sent home. Schools might be closed. Evacuation orders
may be issued. Town mayors and, importantly, state governors all go on TV to
talk about these preparations. The local executives get to do their lord-and-
master thing. If the storm hits, they looks like heroes. If it doesn't, they
still look like dutiful public servants concerned for the safety of their
public. Some people enjoy a good disaster rehearsal, especially if they get to
be the one in charge.

Imho the private sector, specifically the insurance, travel and tourism
industries are the ones who want accurate models. That isn't to say that
prediction should be privatized. The government is probably in the better
place to handle forecasting. But the demand on politicians for better
models/funding arises from industry, not any noble desire to avoid widespread
emergency declarations.

~~~
ghaff
And if you do the disaster rehearsal thing too many times without anything
happening, i.e. cry wolf, the next time no one pays any attention, Katrina or
Sandy hits, and a lot of people die that didn't need to.

~~~
sandworm101
And then the politicians go on TV and lament that people should better listen
to authorities, that they were the wise ones. My point is that an
unpredictable offshore threat advantages local elected officials. Their
careers are advanced by both the false alarms and the disasters.

~~~
ghaff
That is complete and utter nonsense. No one is basically ever happy with the
government response to severe weather events. Leave aside hurricanes. Ask any
number of Massachusetts officials who are no longer in their positions because
of transit and other failures during blizzards last winter.

~~~
sandworm101
How many elected officials in executive branches were replaced? Functionaries,
non-elected government employees, are often wrongly blamed. It is the elected
reps, specifically those in the executive branch (ie Chris Christie) who turn
the unpredictability of storms to their advantage.

