
All 500 of the world's top 500 supercomputers are running Linux - doener
http://www.zdnet.com/article/linux-totally-dominates-supercomputers/
======
b_emery
A win for open source. I was just thinking the other day that it's a shame
that Linus is not a billionaire. Seems he's doing just fine however:

"Finnish-American software engineer and hacker Linus Torvalds has as estimated
net worth of $150 million and an estimated annual salary of $10 million. He
earned his net worth as the principal force behind the development of the
Linux kernel."

Source: [https://www.therichest.com/celebnetworth/celebrity-
business/...](https://www.therichest.com/celebnetworth/celebrity-
business/tech-millionaire/linus-torvalds-net-worth/)

~~~
zanny
I don't think its so much a shame that Linus isn't a billionaire than it is we
have billionaires at all.

There is limited rational or ethical reason for one human being to accumulate
so much centralized wealth. There is being rewarded for hard work / skill /
innovation and there is exploiting people to become rich, and the line between
those is very hard to distinguish when talking about individuals making more
than 9 figures off of the endeavor.

It is also worth considering you don't make billions operating a nonprofit or
making a valuable software product (or any product, in general). You make
billions founding or investing early into a multinational corporation that
amasses more wealth than many nations.

And that is where I think the conversation breaks down - people take personal
offense to conversations about wealth and equality but we aren't talking about
people wielding the influence of themselves _as_ individuals, we are talking
about people who control more assets than some nations, that can potentially
have financial influence on the world stage greater than the combined
influence of millions of fellow humans.

~~~
JoshTriplett
> There is limited rational or ethical reason for one human being to
> accumulate so much centralized wealth

First: yes, there _are_ ethical reasons to accumulate so many _resources_ :
because some problems are incredibly hard to solve and need a lot of resources
to even start to solve.

Second, it is not on the people who have earned something to prove why they
should be "permitted" to keep it; it's on the people who believe it should be
taken away from them to justify _that_. Saying "it's a shame that there are
billionaires" is a complaint that some people are wildly successful and
shouldn't be, or that they shouldn't be permitted to keep what they earn.
There is justifiable resentment and offense to a conversation that starts with
a thinly veiled demand for justification, rather than a presentation of
justification. The burden of proof is approached the wrong way, and nobody
seems to discuss _that_.

(I'm not interested in quibbles over the word "earn" here.)

It's not a shame that there exist billionaires. It's a shame that there exist
people with nothing. The former doesn't need fixing; the latter absolutely
does.

~~~
nhaehnle
> _Second, it is not on the people who have earned something to prove why they
> should be "permitted" to keep it; it's on the people who believe it should
> be taken away from them to justify that._

This is a great example of status quo bias.

You believe that the onus of arguing ethics is on the people who want to
change the status quo when really, we should start by thinking about what
society ought to look like from first principles. And then you absolutely do
have to give arguments for why you want _any_ inequality to exist in the first
place, let alone billionaires.[0]

> _It 's not a shame that there exist billionaires. It's a shame that there
> exist people with nothing. The former doesn't need fixing; the latter
> absolutely does._

While I'd agree that the latter problem is more urgent, the former can't be
dismissed so easily, either. For one, the existence of billionaires (and
upper-range millionaires, really) is pretty antithetical to one of the
foundational principles of democracy, which is "one person one vote". It gets
even worse when you consider the possibility and practice of inheritance.

And besides, some people might argue that if we didn't have the former
problem, it might be easier to tackle the latter -- because when there is less
inequality, there tends to be less focus on wasteful jockeying for social
status overall, and then people might be more inclined to help those that are
below themselves on the social ladder.

[0] There are good arguments in favor of some inequality, but the _degree_ of
it is pretty important.

~~~
unethical_ban
Equality of resources and status, or equality of opportunity?

Some people are worse than others. Some people are less useful to society than
others, though that doesn't make them worse. Some people, given large amounts
of money, can do amazing things for the world. On a whim, I can think of Elon
Musk. If he weren't allowed to have his fortune from Paypal because inequality
is unethical... who would have gotten that money? A government? What would
they have done with that money? And why is government/the public always the
best decider of how wealth is spent?

Your argument may be reasonable in degrees: Who needs $500 billion dollars?
Should there be a legal requirement to invest that money in the public at a
certain point, if it isn't taxed directly? I don't know. But I believe
inequality is ethical and necessary.

~~~
nhaehnle
_Equality of resources and status, or equality of opportunity?_

Both, for two main groups of reason.

The first one is that when the available positions in society are highly
unequal, then equality of opportunity may simply not be enough for a fair
outcome, as who ends up in the comfortable and high-status position ends up
largely being down to mere luck.

The second one is that (at least as long as families and inheritance are a
thing) the two are linked anyway. Inequality of resources and status is the
leading cause of inequality of opportunity, after all. You basically have to
either go Brave New World or increase equality of resources in generation N in
order to make serious progress towards equality of opportunity in generation
N+1.

And we agree that a lot of it is about a matter of degree -- I've said as much
in my original post, after all :)

------
wmf
Note that BlueGene/Q is commonly reported to run "Linux" but 99% of the nodes
are running CNK instead.
[https://computing.llnl.gov/tutorials/bgq/#Software](https://computing.llnl.gov/tutorials/bgq/#Software)

~~~
bantunes
> This is an open source, Linux-like, light-weight, 64-bit operating system
> that runs on the compute nodes

Seems to be a Linux kernel fork with performance optimizations.

EDIT: I stand corrected
[https://en.wikipedia.org/wiki/CNK_operating_system](https://en.wikipedia.org/wiki/CNK_operating_system)

------
tammer

      With an anti-science regime in charge of the government, America will only continue to see its technological lead decline.
    

Brutal to be confronted with such a cutting opinion in an otherwise congenial
article and to have no thoughts of disagreement.

~~~
erikpukinskis
America is a bet that chaos has some advantages over good governance.

~~~
oldsklgdfth
Is good governance obtainable in a country of such scale and idological
diversity?

------
icc97
I remember Linus mentioning [0] about the major benefits of tuning Linux for
super computers that came to be really useful for when trying to get
performance out of a mobile phones decades later.

[0]:
[https://m.youtube.com/watch?v=MShbP3OpASA&t=39m](https://m.youtube.com/watch?v=MShbP3OpASA&t=39m)

------
linarism
Serious question (not in IT), what else can they run?

~~~
fencepost
They could likely run any of the BSD variants. Some form of commercial RTOS
might also be viable. Even Minix might be an option (hire away some of those
Intel folks!) though I'm not sure of its hardware support.

That said, why bother? Linux is customizable enough to strip out almost
everything extraneous for nodes and has the largest pool of experienced
developers, and it's unlikely that any of the other options would have any
performance advantage - and less likely that they could _keep_ any performance
advantage once it was identified.

~~~
caf
Why on earth would you want a RTOS for HPC?

RT is all about privileging latency over throughput, whereas in HPC throughput
is everything.

~~~
dTal
Perhaps because a lot of tasks are not embarassingly parallel, which means
moving data around, which means network interconnects, which means latency
matters. Every millisecond delay in forwarding a packet is thousands of
instructions stalled.

I'm not saying this is actually a sensible tradeoff for any extant HPC
architecture, but it _could_ be. You certainly want the 'OS' that runs the
pipeline on your CPU to be real time!

~~~
caf
It is a misconception that RT means "lowest possible latency".

It actually means "latency with a known upper bound". The point of it is to be
able to give engineering guarantees - the actuators on the rocket engine vanes
_will_ move within N milliseconds of the input voltage from the gyroscope
module changing; the robot arm motor _will_ be de-energised within N
milliseconds of the laser perimeter sensor activating.

Bounding the worst case can often mean making trade-offs in the best case, and
that doesn't make sense in HPC because the hard upper bound just isn't
necessary in this environment.

------
eggy
Where does that put Minix if the supercomputers have the ME chip? Based on the
findings this past month, Minix is on the most computers.

------
wideem
The real question is are they all runix minix(Intel ME)

~~~
vortico
Most of them run Xeon, so yes, but about 10% use either Opteron, SPARC64,
Power, and Sunway.

------
deathtrader666
How does one go about learning more on this topic -- being a Linux admin for a
supercomputer?

~~~
jabl
I don't think there's a degree for supercomputer admin, it's mostly on the job
training. And to be fair, a lot of it is similar to generic Linux sysadmin
stuff. Especially so nowadays with the focus on devops/infrastructure-as-
code/etc., or generally the idea of how to administer many nodes in a scalable
fashion.

If you want to play around, go to
[http://openhpc.community/](http://openhpc.community/) , fire up a few virtual
machines, follow the installation instructions, and start playing around.

------
ForFreedom
These linux OS are customized, compiled and not out of the box, correct?

~~~
pjvandehaar
At
[https://www.top500.org/statistics/overtime/](https://www.top500.org/statistics/overtime/)
click "Operating System" and "Performance Share" to see:

    
    
      - 300 PFlops: Linux
      - 260 PFlops: other 
      - 150 PFlops: Cray Linux
      - 130 PFlops: CentOS
      - 1 PFlop: bullx supercomputer suite (supports RHEL/SUSE/etc)

------
krylon
I vaguely remember that Microsoft offered version of Windows Server for HPC
environments at one point.

Are the reasons for not using Windows purely financial (in a cluster with
thousands of nodes the licensing costs can become substantial) or are there
technical reasons as well?

~~~
microcolonel
I doubt licensing costs have much to do with it, it seems like flexibility
would be the driving factor. Supercomputing centers, vendors, institutions
which use supercomputers, etc. all frequently modify and contribute
modifications to the Linux kernel and associated userland. You have to imagine
they are making changes which Microsoft could not afford to care much about,
nor turn around in competitive time.

~~~
krylon
That makes sense. Thank you!

------
PeachPlum
Ronald Minnich (if you don't know him see [1]) comments :

[1] [https://share-
ng.sandia.gov/news/resources/news_releases/san...](https://share-
ng.sandia.gov/news/resources/news_releases/sandia-computer-scientists-
successfully-boot-one-million-linux-kernels-as-virtual-machines/)

\-----------------

"The semiannual TOP500 Supercomputer List was released yesterday. It also
shows that China now claims 202 systems within the TOP500, while the United
States claims 143 systems."

My fellow Americans:

Did you ever want to directly experience the transition to being a Tier 2
country? This is what it's like. It starts with the cancellation of the
collider in Texas in 1994 (see Tyson's comments on that one). It continues as
steady decline through the next two decades, under both Republicans and
Democrats, and now it is reaching its culmination in the Trump administration
with the replacement of scientists by political hacks. It's not fun. It's
quite sad to watch, that's for sure.

~~~
pcunite
Trump is not the problem. It is all the non-elected dark figures behind it
all.

~~~
mr_toad
From what I can see anti-science, anti-technology attitude is endemic to
modern English speaking western culture.

Elected and un-elected officials are just a symptom of the problem. They'd all
flip-flop if they thought the political and media climate was shifting.

------
bassman9000
_With an anti-science regime in charge of the government, America will only
continue to see its technological lead decline._

This is idiotic. The new admin could well commission new supercomputers for
espionage or other military purposes.

The constant politicization of every single aspect of life is going too far.

------
dxxvi
"For example, the new Linux 4.14 enables supercomputers to use Heterogeneous
Memory Management (HMM). This enables GPUs and CPUs to access a process's
shared address space. Exactly 102 of the Top500 are using now using GPU
accelerator/co-processor technology. All of these will perform better, thanks
to HMM": it sounds like some of the 500 supercomputers use Linux 4.14. I guess
none of them uses Linux 4.14 yet.

------
EthereumDublin
I think it's hilarious that so many run CentOS instead of RHEL. Redhat has
managed to make their product worse than free with their licensing bullshit.

~~~
caf
It makes sense though, because a supercomputer installation is exactly the
kind of place you'd expect to have its own experts that support and customise
the OS.

And there really aren't that many supercomputer installations, so any that
RHEL does have are probably more for the prestige than the revenue.

