
ITX-sized 7-node ARM-64 cluster - rbanffy
https://forum.pine64.org/showthread.php?tid=5461
======
bringtheaction
The Linux support of Pine64 was terrible when it came out. Is this still the
case?

I bought one without looking too closely into it after seeing it mentioned
somewhere online by someone who said it was cheap and it was ARM64. Only after
it'd arrived did I learn that it had very poor Linux support. I've got mine
laying around without ever having bothered to even boot it due to this.

For example, here's a key quote from what Hackaday had to say about the Linux
support of the Pine64:

> The Ubuntu experience was tremendously slow on the Pine64 and I suffered
> several reboots. As of this writing, I have tested all of the software
> distributions on the Pine64 wiki. Only the Ubuntu distribution works poorly,
> and right now I consider the Pine64 to be a waste of $15.

[https://hackaday.com/2016/04/21/pine64-the-un-
review/](https://hackaday.com/2016/04/21/pine64-the-un-review/)

Has Linux support improved recently? Anyone here doing anything useful with a
Pine64?

~~~
itdaniher
The situation's gotten much better since Armbian started building images[1],
but it's still running a stranded kernel for now. This is expected[2] to
change within the next few months. I've heard on IRC of people successfully
running mainline Linux on the A64 SOC, but I don't believe it's recommended.

As far as China single-board computers go, I'm pretty pleased with Pine's
build quality and attention to detail with regards to electrical design that's
reflected in their schematics. Community-provided Linux support has come a
long way in the last year+, it's worth checking out.

[1] [http://linux-sunxi.org/Linux_mainlining_effort](http://linux-
sunxi.org/Linux_mainlining_effort) [2]
[https://www.armbian.com/pine64](https://www.armbian.com/pine64)

~~~
lukasz_Pine64
Moreover, ayufan's images (still BSP) are very solid and contain all the
features you'd want and expect:[https://github.com/ayufan-pine64/linux-
build/releases](https://github.com/ayufan-pine64/linux-build/releases)

As a side-note, armbian's mainline is perfectly usable for headless
applications.

------
trendia
Not to diminish this feat, but I wonder how the Phoronix benchmarks compare
between an ARM cluster and a single x86 chip. Consider the OpenSSL benchmark.

A Sopine A64 compute module costs $29 and performs 15.05 OpenSSL signs /
second [0].

An AMD Ryzen 1700 costs ~$300 and performs 986.73 OpenSSL signs / second [1].

So, if you spent $300 on 10 Pine64's, you would achieve approximately 150
signs / second, but for the same cost you could just buy the AMD chip and
achieve approximately 6.5x better performance for the dollar.

[0]
[https://www.phoronix.com/scan.php?page=article&item=pine-64-...](https://www.phoronix.com/scan.php?page=article&item=pine-64-benchmark&num=3)

[1] [https://www.phoronix.com/scan.php?page=article&item=amd-
ryze...](https://www.phoronix.com/scan.php?page=article&item=amd-
ryzen-1700&num=6)

~~~
pkhagah
I don't think openssl in phronix benchmarks use armv8 instructions, which
makes those benchmarks unfair. There are benchmarks on cloudfare with openssl-
devel, with hardware armv8, where arm performed decently. We have to wait and
see how this pine64 does.

[https://blog.cloudflare.com/arm-takes-wing/](https://blog.cloudflare.com/arm-
takes-wing/)

~~~
revelation
armv7 and armv8 is just 64 bit, so I guess a corner case like openssl signs
could benefit massively (lots of bigint) but normal workloads only marginally,
while taking a massive hit on memory (don't want the 1 GiB model).

The ARM they tested at Cloudflare is also of course nothing like the mobile
chip that powers the pine.

~~~
theresistor
AArch64 is actually a completely new and much improved instruction set. More
registers, less warts that inhibit out-of-order execution, etc.

------
dboreham
Interesting to see a thing I worked on 30 years ago come back around:
[https://en.m.wikipedia.org/wiki/Transputer#TRAMs](https://en.m.wikipedia.org/wiki/Transputer#TRAMs)

~~~
pstuart
I remember the coverage in Byte and was convinced it was gonna go big...

~~~
rbanffy
Yup. I always say it's disappointing we are all running the bastard offspring
of either Unix or VMS on overgrown IBM 5150 PC's featuring enhanced 8080's at
their core ;-)

That now it's possible to have a computer whose sole processor is a Xeon Phi
fills my heard with a tiny little bit of hope.

------
subway
I've recently been fantasizing about something like this, but in laptop form.
Have a dedicated console/display node, a few compute nodes, a 'router' node,
and a gige (or 10gbe) fabric connecting them.

~~~
rbanffy
With a 32 GB of RAM and a beefy x86, you can have a fairly decent cluster
inside your laptop. Even a fully configured Macbook will probably be able to
host a dozen gigabyte-sized nodes, if you refrain from running Slack and
iTunes.

~~~
subway
Sure, and I have that in each of the half-dozen ThinkPads laying about my
place.

That doesn't negate a desire for a unique architecture portable.

edit: Particularly of interest to me is being able to isolate nodes. Your
router and console are on the same physical board perhaps, but only
interconnect via an onboard Ethernet switch (itself managed as a 'slave' under
either the router or console node, depending on where one landed
philosophically). Compute nodes could be powered off entirely when not in use.

~~~
rbanffy
I think a laptop with a server-grade ARM processor would be a lot of fun.

------
digi_owl
Seeing them stacked up like DIMMs make me think of the S100 backpane
popularized by the Altair.

[https://en.wikipedia.org/wiki/S-100_bus](https://en.wikipedia.org/wiki/S-100_bus)

Makes a guy somewhat nostalgic of when computers were a collection of boards
rather than a single chip.

------
PascLeRasc
I wonder how these could do in a homelab environment with each node running a
different VM for pihole/SMB/transmission/etc.

~~~
rootw0rm
why add the overhead of a VM?

------
jand
> To not waste the 24W on the 12V I will probably use the DC/DC converter in
> the pictures and go from 12V to 5V for efficiency reasons...

The more common converters "burn" the voltage difference times ampere. So 14W
will be emitted as heat by the converter, a matching heatsink would be
somewhat important.

Instead of 24W wasted it is now 14W wasted. I am not sure if this is worth the
effort.

~~~
jacquesm
Linear regulators went the way of the Dodo long ago for applications like
this. HF DC-DC converters are the way to go, at the currents involved here
you'd be paying more for the heatsink in a linear setup than for the whole DC-
DC converter.

------
floatboth
Did anyone make similar boards for RPi Compute Modules?

