
IBM Preps Power9 for AI and HPC Launch, Forges Big NUMA Iron - rbanffy
https://www.nextplatform.com/2017/10/16/ibm-preps-power9-ai-hpc-launch-forges-big-numa-iron/
======
kev009
I got IBM to fund the FreeBSD port, they will be at the upcoming devsummit
[https://wiki.freebsd.org/VendorSummit/201711](https://wiki.freebsd.org/VendorSummit/201711).
My interest is in commercial workloads.. the memory and I/O bandwidth are
right where they need to be, and everything else about the uarch is gravy (ISA
is quite nice for network processing, byte swapping load/stores, nice short
fat pipeline, massive cache)

Pricing is actually quite good with the SuperMicro POWER9 systems, especially
against Intel's "scalable processor" family regression where you pay a 4 or 8
socket CPU tax for 2 socket Intel SPP Gold and Platinum configs.

I think POWER9 will be both the first and most cost effective way to reach
400gbps content delivery in a 1u platform.

~~~
throwawaysml
That's cool, may I ask how you persuaded them to bring the FreeBSD port up to
date?

~~~
kev009
Your other comment about NFLX makes you seem suspiciously and hilariously like
a sock-puppet account so I can't actually be candid on that now.

For getting vendors to do things, it helps that I have a 8-figure HW spend.
I'm not GOOG/FB, but it's enough in what vendors call "cloud services
provider" tally that their sales orgs are very hungry for. I have great and
fledgling relationships with Intel, SuperMicro, Dell, Chelsio, Cavium, and now
IBM in the context of FreeBSD for high throughput scale out workloads.

~~~
throwawaysml
> Your other comment about NFLX makes you seem suspiciously and hilariously
> like a sock-puppet account so I can't actually be candid on that now.

I really wondered why they didn't mention POWER at all two weeks ago in the
blog post. And now this post allowed me to raise the question. I'm curious
what you think I represent for me to be a sock-puppet.

I cannot prove it, obviously, but this is just a throwaway account of a single
person who doesn't represent anyone and definitely isn't paid to participate
on Hacker News. Neither am I employed by any large or well-known organization
to have some kind of intent. I'm just a programmer who wishes we had more
POWER machines just like Raptor is trying to achieve.

I did feel uncomfortable asking the question as it might be obtrusive and
thought I managed to word it defensively enough. Sorry, looks like I failed
:).

> For getting vendors to do things, it helps that I have a 8-figure HW spend.

Thanks for answering!

That's encouraging to hear, although sounds like I would need to be the
operator of a video hosting or cdn network or HPC cluster before I could
request a consultation with engineering executives. I'm most excited you have
good relations with Chelsio since I love their NICs and FreeBSD drivers. I
couldn't care less about Intel NICs tbh.

~~~
kev009
Mainly because I have been trying to sign them up with similar reasoning. A
lot of people spent foolish amounts of capital on arm64 enablement, I'm glad
FreeBSD has it for embeded stuff, but the server-class silicon just sucks
compared to intel, amd, and P9. Follow along and watch in 2018 :)

~~~
throwawaysml
Hah, I never understood the appeal of AARCH64 either. It's not like it's some
super clean ISA or that they won't run into similar power-vs-IPC conumdrums
when they start to create x86 competitive server CPUs.

I've always been fascinated by the ARM and PPC embedded support on FreeBSD,
which reflects the real market and not some fantasy land. Which also explains
the sad state of the POWER port right now.

I'm fairly certain that RISC-V will eat ARM and it already started on the
bottom end with embedded chips being replaced with RISC-V. nVidia, storage
(SSD, HDD) vendors already have RISC-V silicon deeply embedded, hidden in
sight. I feel like it will be the FOSS "revolution" all over and more progress
because there's no need to pay stupendous amounts of licensing fees to ARM
Inc. One could have done this with OpenSPARC too, but for political reasons or
technical limitations of the ISA, it never took off.

My prediction is that many embedded ARM chips will be replaced by RISC-V. It
won't be free because someone has to create the desired chip IP, unless you're
happy with the free ones, but there will be competition that doesn't involve a
parent licensing agency grabbing all the money for an ISA they invented 30
years ago. Good times to be a compiler codegen developer :).

~~~
carterschonwald
Woah: there’s little risc v’s in wide spread heardware hidden in plain sight?

~~~
throwawaysml
These things are very hard to find concrete data about for outsider, which I
am one of, but at least nVidia has published that they're building their new
Falcon controller chips based on a custom RISC-V architecture. It's hard to
tell how widespread it is since they are not buying chips from Renesas or MIPS
and looking at the board wouldn't necessarily show you there's a RISC-V chip
in there. All we need now is a high performance variant of lowRISC to combine
it with its safety features, port seL4 and MirageOS to it. Then would get
finally get back what we had with the old Burroughs machines and enjoy safety
and performance without having to choose.

Anyway, they announced this publicly in 2016 so I assume it's already hidden
in some nVidia hardware. I also read about a storage vendor (Seagate? or WD?)
working on a transition to RISC-V, but I cannot find it right now.

[https://riscv.org/wp-
content/uploads/2016/07/Tue1100_Nvidia_...](https://riscv.org/wp-
content/uploads/2016/07/Tue1100_Nvidia_RISCV_Story_V2.pdf)

The reasons cited sound like those we heard about Linux and Samba or Linux and
Apache in the past.

------
gok
"Oak Ridge is getting around 4,600 Witherspoon nodes to crest above 200
petaflops of peak performance with Summit, most of which comes from the six
“Volta” GPU accelerators in the system…"

The CPUs in HPC increasingly exist primarily to keep the GPUs fed. I'm curious
why they don't target having simpler/cheaper CPUs that can be moved closer to
the GPUs. Throw away stuff like SIMD units; anything that can be done there
would be better done on the massively parallel GPU.

~~~
justin66
> I'm curious why they don't target having simpler/cheaper CPUs that can be
> moved closer to the GPUs.

You can certainly buy silicon that's set up that way from Nvidia.

~~~
torpcoms
Can you provide a link? Is it an ARM system?

~~~
justin66
The Parker SoC that comes with the Jetson TX2 came to mind immediately, since
I've got one:

[https://www.hotchips.org/wp-
content/uploads/hc_archives/hc28...](https://www.hotchips.org/wp-
content/uploads/hc_archives/hc28/HC28.22-Monday-Epub/HC28.22.30-Low-Power-
Epub/HC28.22.322-Tegra-Parker-AndiSkende-NVIDIA-v01.pdf)
[https://elinux.org/Jetson_TX2#Processing_Components](https://elinux.org/Jetson_TX2#Processing_Components)

Six different ARM CPUs (two different kinds!) on the same silicon as a fairly
powerful GPU.

------
throwawaysml
When the recent Netflix FreeBSD kernel optimization article was posted, all I
could think of was why they didn't use a POWER machine for its higher memory
throughput. Same would be true for SPARC, but that hardware is even less
available and I wouldn't trust Oracle with my money or engineering effort.

Then I checked to see FreeBSD's POWER port and it didn't look as readily
supported as Linux is, so I assumed that's one reason. Contrary to SPARC,
POWER servers can be had from more than the main vendor and it's the only high
performance server where you have a chance to get the source code for the
microcode and firmware in auxiliary mainboard chips, leaving just the GPU and
NIC. This alone I would think would be a reason for safety-conscious users to
consider and buy it instead of trying to disable Intel ME.

I still think Netflix ought to evaluate POWER9 (POWER10) and possibly improve
the FreeBSD port, and would love more industry support so that little teams
like mine stand a chance to get reasonably priced POWER9+ servers.

The completely free aspect is one reason I believe Kollab is using it for
their servers.

~~~
vvladymyrov
>why Netflix didn't use a POWER machine for its higher memory throughput.

Probably because there is no POWER9/10 option available from AWS and Netflix
is a heavy user of AWS.

~~~
wickberg
The Netflix Open Connect appliances are custom-built hardware they send out to
ISPs. [1]

As evidenced by their heroic efforts to tunes performance[2], and their
careful choice of hardware[3], they could certainly deploy POWER if they chose
to.

[1] [https://openconnect.netflix.com/en/](https://openconnect.netflix.com/en/)

[2] [https://medium.com/netflix-
techblog/cdb51dda3b99](https://medium.com/netflix-techblog/cdb51dda3b99)

[3]
[https://openconnect.netflix.com/en/hardware/](https://openconnect.netflix.com/en/hardware/)

------
throwawaysml
Speaking of NUMA, I'm glad that AMD Threadrypper will force more developers to
get familiar with it, since it's the first mainstream single-socket system
that uses it. It will bring it to the attention of more developers than just
those who tested regularly on multi-socket machines of their employers.

------
throwawaysml
BTW, the new POWER chips can run in a new endiannes mode and therefore lift
the big-endian requirement, if that's a concern of any code base.

~~~
torpcoms
POWER and PowerPC chips have been bi-endian for a while. Since POWER3 would be
my guess.

------
dman
Will ordinary programmers be able to buy one?

~~~
duality
Sadly, it's only for extraordinary programmers.

~~~
throwawaysml
I don't know, I mean, programmers spend so much money and little value
hardware just for brand value and such, so the money is there. But those that
spend that amount of money on computers are seldom in the group of those who
have an interested in a POWER machine.

