
Revisiting how we put together Linux systems (2014) - mondoshawan
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html
======
DyslexicAtheist
> _When a developer wants to develop against a specific runtime he installs
> the right framework, and then temporarily transitions into a name space
> where /usris mounted from the framework sub-volume, and /home/$USER from his
> own home directory. In this name space he then runs his build commands. He
> can build in multiple name spaces at the same time, if he intends to builds
> software for multiple runtimes or architectures at the same time._

it reminds me of the Clearcase version control system. You would run
`cleartool setview <viewname>` to enter the namespace of that "view". Like Las
Vegas, everything you did within that view would stay in that view.

This was before containers and VM's became widespread. It felt a like magic,
because you could run one command and see a totally different build and
tooling environment. Under the hood it meant that running `make` would pull in
objects from other teams who used exactly the same view as you (so like ccache
but across the network scaled to N-engineers).

Most companies have thrown clearcase out when svn/git came along because poor
administration practices could push license / storage costs through the roof
(and it was never cheap to begin with).

We have git/mercurial,... today which I'm quite happy for. But considering how
old all these ideas are, it's a shame FOSS is still stuck in the stone age in
this regard.

[1]
[https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com...](https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com.ibm.rational.clearcase.cc_ref.doc/topics/ct_setview.htm)

~~~
pjc50
And yet ClearCase was notorious for being the bad old world that everyone
wanted to get away from. I'm sure someone will be along with war stories to
tell us why the reality wasn't so great.

~~~
DyslexicAtheist
:) I was one of those who fought hard for many years so that we could replace
it with svn/git, and we eventually did. It was still ahead of it's time back
then. But I agree it was a nightmare to administrate and if you used clearcase
UCM it was one huge horror show for users. UCM meant your whole process is
locked into Rational/IBM's way of doing things (and it had to die because
Agile came along so clearcase felt very outdated).

------
dang
A thread from 2017:
[https://news.ycombinator.com/item?id=13782786](https://news.ycombinator.com/item?id=13782786)

2015:
[https://news.ycombinator.com/item?id=9477612](https://news.ycombinator.com/item?id=9477612)

Discussed at the time:
[https://news.ycombinator.com/item?id=8251288](https://news.ycombinator.com/item?id=8251288)

~~~
dijit
This is super interesting, reading the comments you can easily see the
sentiment steadily getting less hostile to Poetterings ideas.

I guess the only thing you need is total control of all major distros to
change sentiment.

------
detaro
(2014)

------
mongol
Were these ideas ever picked up?

~~~
the_why_of_y
The proposal talks a lot about using btrfs features, but there doesn't appear
to be anything based on that.

However if you look at the design goals, most of them have been implemented,
although there is not yet wide usage.

The best way to try this out in a runable desktop OS is probably Silverblue.

[https://silverblue.fedoraproject.org/](https://silverblue.fedoraproject.org/)

> We want a unified scheme, how we can install and update OS images, user
> apps, runtimes and frameworks.

The storage backend for this is OSTree, using HTTP as the transport; it's
analogous to Git for binary files and does not require special filesystem
features, only hardlinks.

[https://ostree.readthedocs.io/en/latest/](https://ostree.readthedocs.io/en/latest/)

> We want a unified scheme how you can relatively freely mix OS images, apps,
> runtimes and frameworks on the same system.

Flatpak provides the abstraction layer for this; the base OS can be anything
and the applications use only libraries from a Flatpak runtime; by default,
Flatpak uses OSTree to store applications.

Snap is similar, although it requires AppArmor, so can't run on many popular
distros, and I think there's only one runtime that you're required to use, so
no "mix and match".

Docker is similar, although it can't handle desktop applications in a sensible
manner (no sandbox).

> We want a fully trusted system, where cryptographic verification of all
> executed code can be done, all the way to the firmware, as standard feature
> of the system.

There are several points here:

* firmware/pre-boot: [Edit: TPM can be used to verify this, but it's not set up that way by default; Windows 10 Bitlocker can rely on TPM for this]

* bootloader: UEFI Secure Boot handles this

* kernel/initramfs: I'm afraid this isn't typically verified currently, see [https://mjg59.dreamwidth.org/54203.html](https://mjg59.dreamwidth.org/54203.html)

* base OS and Flatpak: OSTree can require GPG signatures for installation; to check the files at runtime, you'd need an additional layer such as dm-verity.

> We want to allow app vendors to write their programs against very specific
> frameworks, under the knowledge that they will end up being executed with
> the exact same set of libraries chosen.

FlatPak SDKs provide this.

Snap has an SDK as well, although note the singular, Snap is exclusively
controlled by one company.

> We want to allow parallel installation of multiple OSes and versions of
> them, multiple runtimes in multiple versions, as well as multiple frameworks
> in multiple versions. And of course, multiple apps in multiple versions.

Flatpak can do multiple framework versions by default, and multiple
application versions with command line flags.

Multiple different OSes hasn't happened yet, probably you'd want at least one
non-experimental OS first...

> We want everything double buffered (or actually n-fold buffered), to ensure
> we can reliably update/rollback versions, in particular to safely do
> automatic updates.

OSTree does this.

> We want a system where updating a runtime, OS, framework, or OS container is
> as simple as adding in a new snapshot and restarting the
> runtime/OS/framework/OS container.

OSTree/Flatpak again.

> We want a system where we can easily instantiate a number of OS instances
> from a single vendor tree, with zero difference for doing this on order to
> be able to boot it on bare metal/VM or as a container.

No idea if this exists anywhere yet.

> We want to enable Linux to have an open scheme that people can use to build
> app markets and similar schemes, not restricted to a specific vendor.

Flatpak/FlatHub solve this; Snap does not because it's exclusively controlled
by one vendor.

~~~
bboozzoo
> Flatpak provides the abstraction layer for this; the base OS can be anything
> and the applications use only libraries from a Flatpak runtime; by default,
> Flatpak uses OSTree to store applications.

Snaps use squashfs, so instead of a hardlink farm, you have self contained fs
images.

> Snap is similar, although it requires AppArmor, so can't run on many popular
> distros,

I'm not sure why you continue to repeat this factually incorrect information.
AppArmor is not required. In fact, guessing you being a Fedora user, you can
verify that by installing snapd from your distro repositories.

However AppArmor is used for providing confinement of snaps (aside from
sandboxing which requires the same interfaces in the kernel as flatpak or
docker do).

> and I think there's only one runtime that you're required to use, so no "mix
> and match".

Currently there are runtimes based on Ubuntu 16.04, 18.04 and upcoming 20.04.
There's some work to support freedesktop runtimes, but it got stuck on
defining metadata for describing security profiles of apps using given
runtime.

> FlatPak SDKs provide this.

> Snap has an SDK as well, although note the singular, Snap is exclusively
> controlled by one company.

Wouldn't call it an SDK, but there's a tool called snapcraft to build snaps in
a convenient manner. You could just as well use 'snap pack' or mksquashfs
directly.

~~~
the_why_of_y
>> Snap is similar, although it requires AppArmor, so can't run on many
popular distros,

> I'm not sure why you continue to repeat this factually incorrect
> information. AppArmor is not required. In fact, guessing you being a Fedora
> user, you can verify that by installing snapd from your distro repositories.

I'm sorry, what I meant to write is, it doesn't run _equally well_ on many
distros; you get a first-class or second-class experience depending on which
LSM your distro has chosen to use. This is not the case with Flatpak, because
it doesn't use any LSM for sandboxing.

Although I've heard there were attempts to enable LSM stacking in the kernel,
which would provide a way to solve the problem; do you know the current status
of that?

~~~
bboozzoo
AFAIK the effort is still ongoing. I believe LWN had some nice coverage on
that, with the last majir pain point being attaching a security context to
sockets in a way that suits all interested LSMs.

Anyways, I would not expect to have LSM stacking enabled in Fedora or RHEL
anytime soon after it lands in mainline, mostly due to political reasons,
thought I'd very much like to be surprised when the day comes.

