
NASA: High Performance Fast Computing Challenge - happy-go-lucky
https://herox.com/HPFCC
======
dzdt
The prizes seem pretty low to me: $15k top prize for several months of work
requiring a skillset that can easily land a 150k-plus-per-year salary. And no
guarantees you get anything for your efforts.

Edit: on the other hand graduate students and postdocs are notoriously
underpaid and often have a lot of freedom in how they spend their summer time.
I guess that is the target competitor pool?

~~~
walshemj
And it doesn't say which Fortran they are using bog standard GFortran or Intel
Fortran with all the parallel extensions.

With this sort of code you also tune it to the machine architecture back when
I did F77 for BT / Dialcom the code was tweaked to suit the PR!ME Architecture
to get extra speed

~~~
jheriko
when they say 'modern' fortran i take it to imply a a compiler and standard
from sometime in the last 20 years.

i've heard people refer to c90 as modern before.. (in the last two years) and
the nasa/astronomy guys are famously clingy to 'known good' tech from the
olden days.

only the very most recent astronomical dataset i acquired was in .csv.
everything is made for fortran from what i've seen over the last 10 years...

------
devrandomguy
> _As a participant, you’ll need to gain access to FUN3D software through an
> application process with the US Government._

This is not at all ready for a competition. The least they could do, if they
want professional volunteer help, is to open-source the platform that everyone
would be deploying on. It was probably built using public money, after all.

Sigh, no shame:

> _FUN3D is: Code developed by the US Government at US taxpayer expense, ..._

~~~
martijn_himself
As a Dutch national living in the UK I did get somewhat excited to at least
have a look at the code. It is always disheartening to read these exclusion
statements for a institution with a global appeal like NASA. At the very least
its bad press for them in a global science community.

I'm also somewhat amused by the claim they expect a 1000x performance
improvement.

------
mynegation
Putting up two TL;DR filters right away: US citizens only, Fortran codebase
:-/

~~~
gravypod
There is sadly a huge amount of Fortran code bases in science. Not old. Not
left over libraries. Not software written for a bygone age.

No... New software. Huge, mission critical, core project, software.

The best part: No documentation, no maintainer, and no migration plan.

Scientists think in terms of how Units of Science per Units of Work.
Translating code, learning new languages, testing, checking correctness,
software validation, bug testing, or even just using external (non-science)
libraries yield a very low science/work ratio.

Just opening a text editor, writing a language you already know, and calling
it "good enough" is high science/work.

Their method of software validation usually boils down to "spit out your data,
graph it, and see if it looks like what you expect".

There's also this notion that "Fortran is fast" that MANY people hold for some
reason. They don't know about what kinds of modern compiler optimizations
they're missing out on and what new IPC frameworks are available. They know
Fortran is fast, MPI is parallel, and that's what they use.

Right now I'm looking at a codebase that is actively being developed and is
written in Fortran. It's about 48,466 lines in total. The source is about 2.3M
in size. No one seems to have formula listings for it. I want to rewrite it
but pulling apart fortran is very difficult.

It's very scary that if I were to write the software in C that it would be
seen as modern! This software is tax-payer funded and included in real time
systems and production software.

(On that note if anyone is good at Fortran and can document the formulas in
this software so I can rewrite them I'd be happy! Please email me)

~~~
qb45
> There's also this notion that "Fortran is fast" that MANY people hold for
> some reason. They don't know about what kinds of modern compiler
> optimizations they're missing out on and what new IPC frameworks are
> available.

Fortran is fast and has no reason to be worse than C. In fact, it may be
easier to write compiler-friendly code in F thanks to "restrict by default".
And are there really HPC-grade C compilers with whatever "modern
optimizations" that don't come with a Fortran frontend? Don't think so.

BTW, I have seen an attempt to rewrite a few dozen KLOC Fortran project to C
and it wasn't pretty. The code was still a bunch of for loops crunching
through a bunch of arrays, just in a different language, and then it had some
new bugs that were introduced in translation and also it produced slightly
different results because of different language rules, compiler and
optimizations translating the same math formulas to different machine code.
And the formulas were written to produce expected output when compiled with,
you guessed it, Fortran.

Don't get me wrong, the old-school Fortran with 3-letter variable names, 72
character columns, GOTOs and stuff sucks donkey balls. But you don't have to
write like that - while I'm no real Fortran expert (sorry), I've heard that
modern standards of Fortran are vastly better and look much closer to C. I
would certainly research this before rewriting everything in C.

~~~
jay-anderson
Also, there are optimizations that can be done with fortran that are more
difficult or cannot be done with C (see
[https://en.wikipedia.org/wiki/Pointer_aliasing](https://en.wikipedia.org/wiki/Pointer_aliasing)).

~~~
qb45
That's what I meant by "restrict by default". You can get Fortran-like
semantics in C by marking function arguments with the _restrict_ keyword, at
the cost of some manual labor (and possibly having to explain this to
scientists who consider computers a necessary evil) ;)

------
jacknews
Hey geeks! NASA!! Aircraft design!! Spaaaace!!!!!

Just optimize our old code, and we'll pay peanuts! If you win, that is.

We were going to hire someone to shephard the old code, but why bother when
you geeks will do it for nearly free because it's so cool!!!

OK I have my cynical hat on, and I can't really blame them for trying a cost-
effective approach, but is this the future of work?

~~~
msl
Well, this _is_ NASA. They are not exactly rolling in cash. Maybe you could
look at it as a sort of a voluntarily paid tax, earmarked for space
exploration? I, for one, would be happy to donate some of my time for the
cause, if they were willing to let me.

> is this the future of work?

Not just the future: many people have contributed to open source projects that
have ended up being used in commercial projects, and never received any
payment for their work.

~~~
jacknews
Good point, though I see open source as something people create for pleasure,
rather than 'solving someone else's problem'. There is much blur between those
two things obviously.

------
soreal
They're giving you a chance to win $15k and all it takes is delivering a 1000x
speedup improvement to software that runs on a 250k CPU cluster?

I'll pass, but hopefully someone else can do this as a labor of love.

~~~
photon-torpedo
> runs on a 250k CPU cluster

Yes, except you are not allowed to run the software on their cluster, instead
you are expected to run it on your own machine.

------
StreamBright
I got really excited until I reached the US citizens only clause. I wish it
was not export regulated, the problem domain sounds pretty interesting.

~~~
georgiev
Why does it always have to be US-citizen only. FeelsBadMan

~~~
rtkwe
Because the difference between a rocket and an ICBM is basically non existent
and being able to simulate the environment inside a rocket engine would help a
lot with designing either. To do pretty much any job in aerospace you'll have
to qualify under ITAR which means you'll have to be a U.S. citizen, lawful
permanent resident of the U.S., protected individual as defined by 8 U.S.C.
1324b(a)(3), or eligible to obtain the required authorizations from the U.S.
Department of State.

~~~
devrandomguy
So, why don't we have these restrictions in Canada, Japan or Europe? Many
countries have advanced aerospace programs. The only reason that I can see for
those export regulations to exist, is because eliminating them would require
effort and PR risk on the part of politicians.

~~~
rtkwe
ITAR is a little old so it's a bit crufty from a different time. But also I'd
be shocked if ESA, JAXA, and CSA were completely open.

------
jxy
Am I reading it correctly? They want people to optimize their code without
directly accessing their hardware. This is not how it should be done.

Is it really from NASA? Are they lacking HPC programmers? Or any one knows
HPC? Either their code is really bad such that optimizing on your home
commodity computer is enough to get it run faster, or they are just making fun
of it.

Even if the code gets much faster with their aging clusters, it wouldn't
necessary be any better with skylake or knights landing.

------
devrandomguy
Is anyone here able to compare this FUN3D CFD solver with open source CFD
software, such as PyFR, Fluidity or OpenFOAM? In particular, has anyone done
any benchmark comparisons?

I'm wondering if maybe NASA would be better off over the long term, if they
ported their models to a more accessible platform. It sounds like they are
having trouble hiring people who can work on their current stack.

~~~
fcanesin
This opensource libraries that you mention (and most in general) are focused
in generic constructs of methods for the solution of partial differential
equations (PDEs). PyFR - flux reconstruction. OpenFOAM - Finite Volume. The
list is actually extensive, there are libraries with focus in Finite Element,
Finite Difference, Boundary Element, Smoothed Particle hydrodynamics and so
on.

The difference here is that this is a solver, not a library. So it is focused
on a specific set of PDEs, by doing so you can apply optimizations that are
specific to this PDEs and the space representation that it uses, this
optimizations are not general to the underlying method of solution (which can
actually be a combination of methods), so it will be extremely hard for this
generic libraries to compete in performance with a focused solver.

------
Twirrim
This will add complexity to the challenge somewhat: The cluster uses a mix of
E5-2680v4 (Broadwell), E5-2680v3 (Haswell), E5-2680v2 (Ivy Bridge), and
E5-2670 (Sandy Bridge) processors.

Each generation has brought different optimisations, tweaked instruction sets,
and performance characteristics. You'll have to optimise for the most common
scenario.

~~~
rootbear
This is a common problem with modern supercomputers, which are all clusters of
some sort. We have a supercomputer at NASA Goddard, much smaller than
Pleiades, that gets new "Scalable Units" every few years. The first parts of
it are now long gone and what's there is a mix of several recent generations
of Intel CPU and some Intel Phis. I'm told that some codes run better on the
older CPUs due to caching and other architectural issues.

------
cube00
Nice, they've worked out how to take away their developer's down time.

 _> To catch integration errors, the suite of codes are repeatedly checked-out
around the clock from our central repository, compiled, and several hundred
regression and unit tests are run. Email and cell phone SMS provide instant
notification of any errors._

[https://fun3d.larc.nasa.gov/chapter-1.html](https://fun3d.larc.nasa.gov/chapter-1.html)

------
lightbendover
Accuracy is only 20% of the puzzle so you could probably grab an easy 80%
score by just returning some random numbers right off the bat. Well, now only
70% because you're not going to get the Originality portion anymore. Even
without the tongue-in-cheek, this seems like a sloppily concocted competition,
but good luck to those who play around with it.

------
m-can
Does the phrase “nonlinear partial differential equations used for unsteady
computations” excite you?

~~~
tanderson92
Yes. But then that's what my Ph.D. thesis area currently is.

------
jheriko
tbh the prizes are well below par. :/

also their us government tie... this is just a coded message for me that i am
not welcome because i am too arabic. at least at first glance. probably its
not but this is my experience of trying to do anything involving america in
the last several years (i can't speak about the trump administration - but i
doubt they are more arab friendly given the bad press).

------
analognoise
So uh...anyone want to team up on this one?

