
LineageOS: Infrastructure Status and Official Builds - loppers92
http://lineageos.org/Infrastructure-Status-and-Official-Builds/
======
lima
How are you going to handle decentralized trust? Anyone who is running a build
slave can potentially compromise the builds, right?

I did offer to host a few build slaves, but I'm somewhat worried about that
and I hope that any potential infrastructure donors will be thoroughly vetted.

~~~
kilburn
Sounds like the perfect excuse to ensure repeatable builds.

You send the build task to n different builders, and compare outputs
afterwards. You could evwn send n build commands and call it done once you
receive k<n identical builds... and you've pretty much solved the reliability
issues with builders

~~~
asdz
What if there're huge number of rogue builders?

~~~
kilburn
You cannot get 100% guarantees, just like you can't even with centralized
builds (what if your build servers get hacked?).

However, you can devise many schemes to improve the probability of builds
being correct. For instance, some kind of "trust tracking system":

1\. You own one builder, which is the only initially trusted source. This is
required to bootstrap the system.

2\. You distribute build requests and collect the resulting builds. Whenever a
builder's build agrees with your trusted source build, that builder gets
"trust points" (up to some _max_trust_ ).

3\. Whenever two builder's builds don't agree, you build that yourself and
compare. The rogue one loses trust points (or gets banned). You can even
retroactively check all other builds from that builder if the requirements are
stringent enough.

4\. A build's trust is the sum of all the builder's trust points that
generated that exact same build from the same request.

5\. You require _k_ times _max_trust_ trust from a built artifact to consider
it valid.

Of course, builders wouldn't know against which other builders will they be
compared. This doesn't make collusion impossible, but raises the bar
significantly because tampering a build when there's a non-colluding builder
involved will get you flagged.

Going further, it even allows for completely segregated systems to track trust
separately. That is, if builders emit signed hashes of their build results,
anyone can keep track of which builders he/she trusts without the need for a
central authority doing so (in a blockchain-y kind of way). This way you could
even have good protection against the "trust authority" itself being hijacked.

I would say that such a system, once well tuned, would be even _more_ reliable
than using single-company tightly-controlled build servers that can be
silently hacked. In any case, it would _surely_ be more trustworthy than just
signing up a few builders that you blindly trust, which seems to be the path
taken here...

~~~
kilburn
Followup: I suspected that this problem should have been studied in the
academic literature. Indeed it is, and this seems to be a good starting point
to read about it, if anyone is interested:

Sabotage-tolerance and trust management in desktop grid computing
[http://estudogeral.sib.uc.pt/jspui/bitstream/10316/4095/1/fi...](http://estudogeral.sib.uc.pt/jspui/bitstream/10316/4095/1/file608a385c50234adaa56526c950cb8332.pdf)

------
anilgulecha
This is an opportunity for a cloud provider to just donate the lineage team
with the needed hardware, which isn't much.

Even assuming 100 supported devices, with 24 builds/day, and a 4x buffer, the
project only need ~16 reserved build machines.

------
uvatbc
I just sent an email to the contact address on the linked LineageOS site but
after reading the latest comments on this story, I thought I might as well
gauge HN's interest in what we're doing in my startup.

We have a platform that is capable of reducing compilation times
significantly. We've already proven it for a few projects: the Linux kernel,
qemu, Qt, etc.

What I'd like to know is if there are people who'd be interested in such a
platform as a service?

------
cowmix
What's preventing them from using a free service like BitBucket Pipelines to
do these builds?

~~~
jeroenhd
I compiled a nightly build of LineageOS yesterday. Building it for just my
phone requires: -8GB of RAM -70GB of disk space

Pulling down the source code from github using my 500mbps connection took
about half an hour. The first build took over 90 minutes. Adding a single file
and rebuilding took 25 minutes with compiler cache enabled.

Presumably the compiler cache will speed up building for multiple devices and
device specific builds will probably only take half an hour after the first
build, but these numbers will not fit in any free plan.

Granted, bitbucket seems to have decent pricing for their Pipelines but the
price for building weekly for all devices will probably be too expensive in
the long run. That's why they're looking for servers donated by the community.

