
Microsoft’s new x86 DataCenter class machines running Windows (2018) - rbanffy
https://www.anandtech.com/show/13522/896-xeon-cores-in-one-pc-microsofts-new-x86-datacenter-class-machines-running-windows
======
aloknnikhil
From one of the comments on the article

> I think Intel / Microsoft have found a way to interconnects scalable and
> this comes interesting thought on value of have more cores on cpu.
> Especially if system was designed to be pluggable and if one of cpu's failed
> than it does not bring entire system down.

I think this is an interesting point. If the QPI links on the Xeon Platinum is
restricted to 3, I wonder what the 32 socket mesh looks like and what
implications that has on availability of the system as a whole if one CPU was
to fail. Also, unless these are fully NUMA aware workloads or each socket was
dedicated to a VM, the latency has to be terrible for memory access especially
if the interconnect is multiple hops away. Clearly it's not going to be a
fully connected mesh?

~~~
freeqaz
This starts to blur the line in my head between the low-level system concepts
that your compiler/OS has to think about and the concepts you have to think
about for distributes systems.

With the hardware systems and interconnects today, we've drawn a line in the
sand based on what we've seen work and fail. Something like Redis is fairly
well understood for many architectures. But when you have crazy NUMA
architectures and different memory models like Intel Optane, what changes?
Where do you draw the "new" line for when to use Redis vs keep everything
"local" to one box?

Will we see a local Kubernetes extension (or new software) that makes building
for "massive boxes" easier?

Kind of a cool idea.

------
sradman
> Back in the days pre-Nehalem, the big eight socket 32-core servers were all
> the rage, but today not so much, and unless a company is willing to spend
> $250k+ (before support contracts or DRAM/NAND) on a single 8-socket system,
> it’s reserved for the big players in town. Today, those are the cloud
> providers.

SAP HANA is a major driver for single servers with the maximum number of cores
and maximum memory per core that Intel supports. This is true for on-premise
as well as Azure [1] and AWS [2] IaaS.

I think of it as commodity scale-up and the superior price-per-performance
applies to any hybrid in-memory column store though HANA seems to be the only
offering that exploits this approach. You can store a lot of data in 24 TB (or
more) of in-memory compressed bitmap indexes.

[1] [https://docs.microsoft.com/en-us/azure/virtual-
machines/work...](https://docs.microsoft.com/en-us/azure/virtual-
machines/workloads/sap/hana-available-skus)

[2]
[https://aws.amazon.com/sap/solutions/saphana/](https://aws.amazon.com/sap/solutions/saphana/)

~~~
fomine3
Fun quote:

> From what we have heard from both OEMs and Intel, the vast majority of
> 8-socket servers today are sold in China. The most common reason for this
> that we hear is not because of the scale-up benefits. Instead, it is because
> of the numerology of the 8. It just so happens that 8 in China is a lucky
> number, much like 7 is considered by many in the US to be a lucky number.

[https://www.servethehome.com/why-all-servers-are-
not-4-socke...](https://www.servethehome.com/why-all-servers-are-not-4-socket-
servers/)

------
blattimwind
The late 1990s called and want their Numalink back.

~~~
p_l
Some of the successful large-socket-count servers, specifically SGI
Ultraviolet series (now sold by HPE AFAIK) actually use NUMAlink - a much
updated version, but the basic remain the same as with MIPS and Itanium
systems.

------
aloknnikhil
This is from 2018. Can we add that to the title?

~~~
dang
Good catch. Added. Thanks!

------
fortran77
Why does HN sometimes truncate leading words/numbers in submissions? The
actual title is "869 Xeon Cores...."

~~~
wmf
Those are common clickbait patterns.

~~~
thoraway1010
No kidding. Despite the HN posters saying it's "factual" this is likely just 4
socket systems tied together with an interconnect.

If intel is really going to market these type of systems, they need to be a
LOT LOT more transparent on interconnect latency / memory management issues
etc.

These headlines are almost always clickbait when you try to run high
contention workloads on these supposedly massive systems.

What is the name of the interconnect approach being used here? That's what I
want to know.

~~~
yjftsjthsd-h
If it runs a single system image, how is it different from any other NUMA
system or a big.little arrangement?

~~~
thoraway1010
Some big.little's have very good interconnect / memory access. Some systems
have remote NUMA memory - you are going off one board over an internconnect to
another board to memory there.

The headline is so misleading. I could wire together a ton of machines over
10mbs, and yes, I could get NUMA going on it and a high core count, but the
performance would be horrendous. Maybe you could even run super NUMA over DB9
9 bit serial ports with remote memory access latency in hours?

------
johnklos
Like lipstick on a pig.

~~~
Koshkin
Thank you for the mental image of a compute stick running lisp.

