
Popcorn Linux – for ISA-diverse multi/many-core architectures - rbanffy
http://www.popcornlinux.org/
======
joshumax
A few months ago I created/worked on a project that involved separating PNaCl
(the portable native client) from the Chromium project, rewriting parts of it
for use as a generic JIT compiler, and building a small PoC distro around it
where all of userspace were PNaCl binaries except for the loader/JIT (which
was a regular old architecture-specific ELF that existed read-only in memory
and loaded by a slightly-patched Linux kernel) Only PNaCl executables were
allowed to run in userspace, which allowed for secure execution of user
binaries thanks to the sandboxing features of the modified portable native
client. However, another benefit of the PNaCl is that binaries are compiled to
a performant architecture-agnostic IL (similar to LLVM's IL) and then quickly
finalized by the PNaCl loader into a regular architecture-specific binary and
loaded into secure memory when they are started.

This meant, in a nutshell, I could have a completely architecture-agnostic
root filesystem, with the only architecture-specific binaries being the kernel
and PNaCl runtime, which sat in the initial ramdisk.

~~~
pjmlp
So basically you recreated IBM's AS/400 mainframe.

Had you researched it for your work?

I would be interested in some kind of comparison.

~~~
joshumax
Wow! I actually wasn't aware that TIMI existed until just now! All I've ever
really known is ARM's work on the Jazelle hardware subsystem. I'll have to
check this out more in the morning though!

------
mmastrac
Whoa - this could be really interesting.

Imagine a hybrid x64/ARM system where you could run simpler, low-intensity
tasks on your low-power ARM core (Twitter clients, Facebook, etc) but more
intense tasks or entire applications (Photoshop, development IDEs, etc) on
your x64-compatible chip.

Or a system where you take your ARM core + storage with you in your phone and
dock it with an x64 core in your desktop.

~~~
pjmlp
This has been typical approach in mainframes and other OSes with bytecode as
executable file format.

AS/400 architecture upgrades are done without any big worries about CPU
backwards compatibility thanks to them being like this.

------
qubex
This reminds me very much of the heterogenous OS TAOS designed for the
Transputer platform and for other hardware processors of varied architectures
coexisting on the same system.
[http://www.dickpountain.co.uk/home/computing/byte-
articles/t...](http://www.dickpountain.co.uk/home/computing/byte-articles/the-
taos-operating-system-1991?hn)

~~~
qubex
Unfortunately it is too late for me to edit my previous post with a link to
more illustrative material:
[http://www.uruk.org/emu/Taos.html](http://www.uruk.org/emu/Taos.html)

------
LarryPage
Such a good name.

------
cgb223
Thought this was for PopcornTime for a minute..

Multi ore Linux could be pretty cool.

Mix and match CPUs anyone?

------
westmeal
I'm not sure I understand. Does this mean that multiple linux kernels are
split up, for example, on the graphics card cores such that each core on the
GPU and CPU gets a "mini-kernel"? And then from that point all programs can
run selectively on certain bits of hardware?

~~~
antobarba
Popcorn Linux main idea started when trying to run Barrelfish OS (from ETHZ)
we asked ourselfs "can I apply the same ideas to Linux"? Then we ended up in
taking a somewhat different route in terms of system design for different
reasons, some were technical. My personal perspective on the project was to
try to understand if I can power up and down heterogeneous cores at any time,
scaling indefinitely in the numbers of cores ... I changed that idea a little
bit since I started the project (2012). The current version of Popcorn Linux
supports aarch64 + x86_64. Popcorn Linux runs only on OS-capable processors
(no GPUs, no FPGAs). The kernel is not split up. You run different instances
of the kernel on different ISA processors - one kernel per processor island
(that is a set of processor cores with homogeneous characteristics). When a
kernel boots up, if it is not the first one, it will join the community of
Popcorn kernels. Operating system services are distributed, therefore things
like memory are globally managed. An applications to run among kernels must be
compiled with the Popcorn Linux toolchain, and must be executed inside the
Popcorn namespace (just an additional Linux namespace), in which the
applications can see all the available memory and CPU cores. When your
applications is in the namespace the only thing that it has to do is to call
sched_setaffinity to migrate to another core - as easy as POSIX/SMP. You can
migrate between different ISA at any point in the code (there is no limitation
such as in offloading, that you have to go back as soon as the offloaded
function is finished) - this is thanks to the stack rewriter (see below). The
current way Popcorn compiles an application to be run on aarch64 + x86_64 is
to produce 2 binaries, one for aarch64 and one for x86_64. For now, the
compiler toolchain is modified to align all symbols in each binary at the same
address, and all data types have the same sizes; all alignments and paddings
are forced the same way. For this reason we used musl, and yes, you need to
recompile each library (although this is not strictly necessary in some
cases). The executable is statically linked. The operating system cannot
cross-ISA migrate your application at arbitrary points, the migration should
be explicit in the code itself (but we added some OS-suggested migration, at
automatically placed migration points). (If you want to cross-ISA migrate with
OS control, you need emulation until a valid migration point, you can check
the work from Venkat + Tullsen.) Finally, because the executable code of
different ISAs sees the address space identically there is no need of state
transformation during migration - this enables the developer to migrate a
single thread, or a subset of threads of your application. The only thing that
Popcorn transforms during cross-ISA migration is the stack, thanks to Rob that
wrote the stack transformation routine. We decided to do not implement a
single stack layout because we believed that the current stack ABI is
optimized for performance, but as far as I remember we never tested the
alternative option.

