
IBM Adds POWER9 AIO, Pushes for an Open Memory-Agnostic Interface - rbanffy
https://fuse.wikichip.org/news/2893/ibm-adds-power9-aio-pushes-for-an-open-memory-agnostic-interface/
======
kev009
IBM has solved another fundamental problem before most are aware of them, the
other CPUs are hitting a memory and interconnect wall. If you're paying
attention, this has been the norm for POWER since its inception. It's too bad
the OpenPOWER stuff has been underused. It's really good stuff.

~~~
nullc
With the exception of talos and various bits of used hardware POWER has only
been available in rather expensive systems-- so no onramp.

Even with things this announced it remains the case that EPYC, not power9, is
the king of memory bandwidth per dollar at least at most price-points.

~~~
rbanffy
> at most price-points

Worth noting POWER9 reaches price points (and profit margins) EPYC server
vendors can only dream of.

Sadly, you are right about onboarding. I'd love to have a POWER9 box as my
desktop workstation, but I simply can't justify the price compared to a beefy
Xeon (or EPYC) box. IBM would be wise to have a POWER9 that could compete with
low-end Xeon servers.

~~~
nullc
Indeed, there is a realm of astronomical hardware that is covered by power9
that you're not finding amd hardware for, but going down to "merely the price
of a new car" and below is where the comparison falls apart.

And the astronomical end really only matters if you need special requirements
like single system images... if you just need lots of cycles or memory
bandwidth, a cluster of lower end hosts (and 40GBE nics) is a lot less pricey.

> a POWER9 that could compete with low-end Xeon servers.

The Talos II is in the vague order of magnitude for that... which is A LOT
better than before it existed.

~~~
rbanffy
True, but it's still, sadly, above my impulse-buy limit.

To make things a bit harder, I live in Europe and there aren't as many sources
of used POWER8+ gear as there are in North America.

------
rhn_mk1
> Compared to the standard direct-attached DDR4, with an OMI DDR4 DIMM, you
> are looking at triple to quadruple the bandwidth at the cost of 5-10 ns
> higher load-to-use than RDIMM.

It would be interesting to see benchmarks on the impact.

~~~
fluffything
> at triple to quadruple the bandwidth at the cost of 5-10 ns higher load-to-
> use

You need to make full use of the bandwidth for this to be an overall win.

~~~
drewg123
Once you use more than 75% of the available DRAM bandwidth, things start to
get wonky, with large unpredictable CPU spikes, as the CPU stalls increase. So
I think you'd see a definite benefit if you use more than 75% of the available
bandwidth.

We (Netflix CDN) are currently bandwidth limited for serving TLS encrypted
video using software ktls. (~100Gb/s on broadwell, 150-160Gb/s on
skylake/cascade lake, 220Gb/s on EPYC Rome). So I'd love to have twice the
bandwidth. With twice the bandwidth, I think we could probably serve 400Gb/s
from a single Rome based box.

Note that I'm also looking forward to trying new kTLS offload NICs that should
cut the bandwidth requirements.. those are a much more viable short-term path
forward.

~~~
e12e
Wow. What kind of networking hardware do you push to/through? I have a hard
time visualizing where/how you would "put" 200 gbps worth of data over (any)
network...

~~~
rasz
as to where, is internet infrastructure this bad in US? Even something like
35C3 had 500gbit pipe last year
[https://media.ccc.de/v/35c3-9576-35c3_infrastructure_review](https://media.ccc.de/v/35c3-9576-35c3_infrastructure_review)

~~~
kev009
No, the US is one of the most densely connected countries in the world. Fiber
was over-provisioned by the functional Bell monopoly, railroads, and a
smattering of upstarts in the 1980s and massive infrastructure capital bubble
in the 2000 dotcom crash which was more about this infrastructure than
software or computer hardware.

If you hear people in the US complaining about their internet connection, it
is because the local duopoly of telephone and cable company is failing them.
In terms of industrial fiber, the competition is fierce and you can light up
Nx100G to almost anywhere in the country.

------
baybal2
10 years ago, when AMD moved high speed memory controller to the CPU, it was
lauded the best thing since the sliced bread.

Now, 10 years later, we again move memory controller off the CPU.

~~~
rbanffy
Things change. What made sense 10 years ago in PCs and midrange x86 servers
doesn't currently make much sense on the higher end of the POWER9 spectrum.

------
saagarjha
> Centaur memory buffer chip

Any relation to Centaur Technologies?

~~~
close04
Doesn't look like it. The Centaur is described as "a memory buffer chip
designed by IBM for their POWER scale-up microprocessors" while VIA owned
Centaur's website seems to have gone from "tune in for a massive unveiling
later this year" a few years ago to what looks like a promotional leaflet for
student and internship programs. I guess they mostly do research stuff now and
very unlikely to design stuff like this for IBM.

~~~
phonon
Zhaoxin has taken over distribution (and possibly some of the development) of
the Centaur designs. Centaur has over 100 employees listed on Linkedin, so I
think they have been working quietly for many years.

[https://en.wikipedia.org/wiki/Zhaoxin](https://en.wikipedia.org/wiki/Zhaoxin)

[https://www.tomshardware.com/news/china-zhaoxin-
kx-6000-core...](https://www.tomshardware.com/news/china-zhaoxin-
kx-6000-core-i5-7400,39694.html)

~~~
rbanffy
I assume they are still an important player in the embedded x86 niche. When I
worked on the Brazilian voting machines, we were using Via CPUS.

~~~
buran77
The problem is as far as anyone can tell they are living off the same designs
they had a decade ago and not actually coming up with anything new. None of
those designs are actually able to compete in any segment these days: not
performance, not low power, not performance/W. Unless they can somehow be the
absolute cheapest option and go for the segment where anything is good enough
it's hard to see them make a living, let alone thrive and come up with
something new.

