
An EPYC NixOS Build Farm - nuriaion
https://grahamc.com/blog/an-epyc-nixos-build-farm
======
tmd83
This is quite surprising. I assumed they would get an EPYC server with higher
core at similar price (due to the cost of the CPU) and get advantage in
performance/dollar. I didn't expect EPYC to beat a similar physical core count
intel server running at similar core count let alone by this much margin. Am I
missing something here?

I'm saying this as someone who really wants AMD to succeed and push the
ecosystem forward. I just want to make sure that I'm reading the results
correctly here.

They are effectively getting 35% extra performance 60% of the price that's
just beyond impressive.

~~~
boris
EPYC is compared to a previous generation Xeon with lower TDP (105w vs
155/170w) and slower RAM (2400 vs 2666). The Xeon server is more expensive
probably because it has 4x the amount of RAM.

Edit: actually, this is a _dual_ Xeon machine so its TDP is slightly higher
than EPYC's.

~~~
yjftsjthsd-h
Doesn't that make it _more_ impressive? The AMD build has a quarter of the RAM
and still came out ahead!?

~~~
wereHamster
Unsurprisingly, the amount of RAM has little effect on a CPU-heavy workload.

------
chris_st
I'm curious if anyone knows the test tool they're using here, in particular
for lines like:

    
    
      $machine->waitForWindow("^Desktop ");
    

It looks incredibly useful.

Thanks!

~~~
O_H_E
Well there seems to be something called nix-shell, but I never heard of it
before. It seams pretty useful. The link to plasma tests is provided in the
article.

[https://github.com/NixOS/nixpkgs/blob/49a6964a4250d98644da61...](https://github.com/NixOS/nixpkgs/blob/49a6964a4250d98644da61f24dcc11ee0b28c4f9/nixos/tests/plasma5.nix)

~~~
ris
This isn't nix-shell. nix-shell is a tool for spawning ephemeral software
environments composed of nix packages, something a little like python's
virtualenvs.

Digging a little bit, nixos' tests appear to be written in perl and are
executed in a context that has access to a bunch of useful primitives for
asserting these sorts of things. [https://nixos.org/nixos/manual/#sec-writing-
nixos-tests](https://nixos.org/nixos/manual/#sec-writing-nixos-tests)

------
bearjaws
For anyone confused by instance types, this is for Packet.net

So these two servers are pretty similar, I originally thought they were
comparing a Amazon m1.xlarge vs EPYC...

------
newnewpdro
This is comparing far more than Xeon vs. EPYC, not even a single mention of
the storage subsystem on these respective machines. It's not like the tasks
being timed are 100% CPU-bound...

~~~
bearjaws
On the other hand, the EPYC server is 50% less per month so I believe it is
showing how much more you can get for less $.

~~~
praseodym
The EPYC server only has a quarter of the RAM though, so it’s really apples
vs. oranges to compare prices directly.

------
davidmr
I wonder if there were NUMA effects at play since the Intel server was dual-
socket and the EPYC was not. It's not entirely surprising that a CPU-heavy
workload reading from filesystem cache (i.e. memory) could cause a substantial
amount of nonlocal reads that the single-socket server wouldn't have to deal
with.

That said, that's just the first thing that popped into my head to explain the
result. Either way, the results here are extremely encouraging!

~~~
wmf
A "single" EPYC is eight 4-core processors so there are even more NUMA effects
there.

~~~
h4b4n3r0
I think it’s the other way around: four 8-core processors

~~~
blattimwind
It's four dice with two groups of cores of four each connected by a callosum.

In NUMA-speak, each group is (probably) a node with neighbouring nodes having
a lower distance.

------
slededit
Unless your buying top of the line Intel parts AMD is going to have more
performance at most price points. It's the only logical decision unless you
use AVX-512 or your workload is single threaded.

------
cjhanks
They are testing performance of a heavily modified and secure Xen type 1
hypervisor against a native computer with unspecified security policies?

This seems like a meaningless measurement.

