
Global Forecast System source code - geetee
http://www.emc.ncep.noaa.gov/GFS/code.php
======
davidwihl
Independently I was just wondering tonight why weather forecasting does not
use some Machine Learning techniques as there is an abundance of training
data. Wouldn't any number of regression ML algorithms find a more accurate
prediction model? If Google did weather....

~~~
darksaints
Sometimes Machine Learning is the wrong approach. Direct mathematical
modeling, if the underlying causal phenomenon are well known, bypasses the
whole learning aspect of an explanatory system. Machine learning might help
make sense of the residual noise that can't be accounted for with direct
mathematical modeling. But at the level of sophistication that I've seen
(predicting wind patterns using Finite Element Analysis methods mapped to
topographical patterns fine-grained enough to model small buildings), I can't
imagine any machine learning methods that could make any sense of the residual
noise.

~~~
davidwihl
I agree that Machine Learning is not always the right approach but wouldn't it
be a relatively inexpensive experiment to see if it could be used for more
accurate predictions? I'm surprised that topographical patterns are mapped
that precisely when ASOS and AWOs stations are so far apart. Also the weather
data from commercial airplanes is so far above the surface that modeling small
buildings would be meaningless.

I heard a talk from ex-Googler climate.com (now Monsanto) guys. Their
forecasts were accurate enough to build a very profitable insurance business.
I bet they used Bayesian rather than FEM techniques.

~~~
bainsfather
Is it that 'physics simulation' forecasting is best for short timescales (e.g.
< 1 week in the uk), whilst ML can be better for estimating next year's
weather statistics in some particular place? (Because your butterfly effect
means that simulating the actual weather out to 1 year 'accurately' is not
possible).

Presumably any short-term ML forecasting would use the physics simulation as
it's main input, and then try to improve slightly on it (e.g. maybe you
observe that in a particular small area, the rainfall is on average 10% more
than the physics simulation, so your ML method would add 10%, giving you a
slight forecast improvement)?

------
zavi
Do they also make raw sensor data available for free? I.e. if I wanted to
build my own forecast system based on this code, I would still need all the
data they collect with weather balloons, right?

~~~
jakethedog
To create a global NWP system from this code you would need observations
(soundings, SST, satellite measurements) as well as a super computer.

~~~
tankenmate
WCOSS (Tide & Gyre): IBM iDataPlex/Intel Sandy Bridge/Linux

208 trillion calculations/sec; 10,048 processing cores; 2,590 trillion bytes
of storage

[xref:
[http://www.emc.ncep.noaa.gov/GFS/doc.php#comopesys](http://www.emc.ncep.noaa.gov/GFS/doc.php#comopesys)]

~~~
jscheel
Here's a really interesting read about the trials of upgrading to WCOSS:
[http://www.washingtonpost.com/blogs/capital-weather-
gang/wp/...](http://www.washingtonpost.com/blogs/capital-weather-
gang/wp/2013/06/25/new-weather-service-supercomputer-faces-chaos/)

~~~
bainsfather
I'd assumed that 'trials' meant 'snafu in a big software project'.

But actually it's about the butterfly effect making it impossible to get exact
agreement between the old and new systems for forecasts longer than about 5
days.

------
noobermin
Fortran 90 is still the language computational stuff is made in today. I'm not
convinced, however, that that choice is really for performance reasons over
cultural reasons.

Still, Fortran 90 is hell of a lot better than Fortran 77, which unfortunately
is the language of choice for some of the people I'm being asked to work with.

~~~
upmygreen
I'm a part of this community. I assure you speed in number crunching is a huge
reason this is written in F90.

~~~
tankenmate
Indeed, automatic (and largely implicit) use of SIMD and OpenMP by the
compiler is very very good with F90 compared to almost all other widely used
languages.

------
kolbe
This is really cool. If I knew the slightest bit about Fortran, I'd be
distracted for weeks.

~~~
bainsfather
imo, fortran90 is about the easiest language to follow - if language is all
that is stopping you (and you know some other language already), then you
should just pile straight in.

From looking at some of the code, what is hard for me to follow is the
'overall view'. The documentation (of the model) looks quite nicely done, but
there must be a huge amount of fluid dynamics etc etc knowledge, that would
take me years to learn.

------
monksy
Why don't they just release this on Github?

Also, it would be rather interesting for a rewrite of this by the open source
committee.

~~~
reubenmorais
Your first sentence sniped me and I almost wrote a Big Response before I read
the second line and realized you're obviously joking…

