

Signing the OPAM Repository: TUF Meets Git - hannesm
http://opam.ocaml.org/blog/Signing-the-opam-repository/

======
justinsb
(Edit: I think it is great that we're working on signing software. I think
that is very important, so important that I want to see it done right. So I
have questions, in particular about TUF vs the more knee-jerk alternatives.
But I would like to see signing, whether through TUF or the more well-known
options.)

It seems that in order to add a new package or developer, a quorum of
administrators must go to their secure signing machines and approve the key
for that delegation. I don't see how that scales to active communities with
lots of new developers.

Despite that administrative work and delay, it doesn't seem that the end
result offers any guarantees about the repository other than that files
haven't been tampered with. The repositories can still include malicious
people or malicious software, because the administrators aren't doing any form
of validation.

I also wonder what happens if people start squatting names and how the
administrators will determine who has the legitimate claim to names.

This feels very similar to the CA system, except that we have pinned a
particular CA (our "trusted administrators"). We have to compare that to
choosing a CA (e.g. LetsEncrypt.org) and requiring that all keys be signed by
that. I'm no fan of the CA system, but it does seem that the CA system (with
pinning) is more tested, and has already implemented stronger procedures and
guarantees than what we are getting here.

The timestamp signing system is nice, in that it allows for easy mirroring
without requiring TLS. Given the timestamp signing key is online, it does seem
we could more easily just serve the timestamp file from a central server
though, over https, and then allow everything else to be mirrored (the
[https://security.debian.org](https://security.debian.org) model, I believe)

~~~
AltGr
Not at all: adding new packages or developers is automatically signed (by the
snapshot bot). Administrators may later sign the keys and delegations,
individually or in batch, after verifying the developer identity out of band
(the specification doesn't yet include the mechanisms for this verification,
but with a git repo based on GitHub, the GitHub login gives a good start).

~~~
justinsb
Thank you for the clarification. Can I install these "not yet trusted"
packages? Presumably I can, I just get a big warning. And how do you deal with
squatters - can I (as the "real developer") submit a second claim on an
unverified package name (that an impostor has claimed)?

Edit: And does this mean that the snapshot bot automatically merges (well-
formed) pull requests for new packages?

~~~
AltGr
The pull-requests for new packages are manually merged (as it the case now),
automatically signed.

Names uniquely identify one package. A repository maintainer can replace the
owner of a package, and clear the existing package contents. This shouldn't be
done for a different package with the same name if any users still have the
package installed, though, as they will consider that an up/down-grade.

Hope that clarifies a little!

~~~
justinsb
That does help me a lot - thanks! The manual "pre-approval process" via
accepting the PR is what I was missing, and that does seem to fix most of the
shortcomings of the underlying system (assuming a reliable human!) So you have
a human that looks at incoming requests to prevent abuse, and to arbitrate
problems. And then the snapshot server signs that result to allow distribution
over HTTP, and then periodically it gets signed by the quorum of "gods" to
indicate that these packages have the full seal of approval.

I think it is pretty specific to OCaml's current workflow and volume, but
signing is better than not signing!

~~~
AltGr
It's indeed specific to our workflow -- allowing additions without moderation
would cause all sort of problems we don't address; but not to OCaml, e.g.
Homebrew uses a very similar, GitHub based workflow (which we actually drew
lots of inspiration from) !

What is nice is that the moderation, if given the proper reports, e.g. Travis
with lint and security checks, is very fast in the simple cases, and readily
opens discussion in the others -- that's where GitHub really shines.

I mentionned GitHub, but note that we intend to also provide the tooling to
host your own signed repository, indenpendently of it.

------
amirmc
For those who are interested in following the discussion that the
maintainers/devs are having, you'll find it starting at:

[http://lists.ocaml.org/pipermail/platform/2015-June/000595.h...](http://lists.ocaml.org/pipermail/platform/2015-June/000595.html)

------
amirmc
I've recently become really interested in things around the periphery of
secure systems keys etc.

For example in this scenario, what happens if one of key-holders dies?
Presumably the remaining are enough for a quorum, but say something bad
happens at a conference that many of them are at? It makes me think of why the
President and Vice-president aren't in the same place at the same time.

In this case, I expect it would be possible to start again and select a new
group of keyholders but what downstream effects would that have? In any case,
I'm looking forward to seeing this progress.

(Incidentally, one of the contributors is also an author of the OCaml TLS
stack.)

~~~
hannesm
The quorum definitively helps. If too few keys/people are remaining, there are
two options: download and install an updated opam (with a new set of keys) out
of band OR using another opam-repository (with e.g. trust-on-first-use) for an
update of opam itself (which then ships new keys for the main opam
repository).

------
allendoerfer
What I do not get about these repositories is, why they have no namespacing or
a built in democratic mechanism to transfer ownership. Because this does not
exist, two related problems occur: Outdated packages squatting their own name
and otherwise worse packages squatting a generic name.

The adoption problems Python 3 still has at this point, could be mitigated
entirely if package owners could lose the name for outdated packages and it
would be transferred to a Python 3 fork. npm has this problem on an even
larger scale. The Python community can at least slowly move away from PIL and
use Pillow instead, because the libraries are reasonably big. This is much
harder to do with all of npms tiny packages. I doubt that the best solution
always wins, because the best name is just an enormous advantage.

If names would be democratically assigned, we would get some kind of extended
standard library. Newbies that want the best library for their problem that
works with the current version of the language would not have to search for
Beautifulsoup, voluptuous or requests, but could just use xml, validation and
http.

Of course, this is a hard problem and you have to be careful. We do not want
Java, we do not want broken dependencies. So my ideas a solution could be
based on are these (example is for pip, but applies to others as well):

    
    
      # No fork exists
      pip install flask
      
      >Downloading/unpacking mitshuko.flask
      >…
    
      # A fork exists
      pip install flask
    
      >Multiple users have created a package named "flask". Please specify the owner
      >of the package or use "--most-popular" to install the one with the most 
      >installations in the last 3 months.
      >
      >*****************************************************************************
      >Attention: pip does not check if the package actually does the same thing, be
      >sure to check that before you switch to a package from another author.
      >*****************************************************************************
      >
      >mitsuhiko.flask (10000 installations)
      >forker.flask (3 installations)
    
      pip install --most-popular flask
      
      >Downloading/unpacking mitsuhiko.flask
      >…
    
      pip install -U flask
    
      # On upgrade, this message is only shown, when the used package is 
      # already at the current version, has not been updated for a long time
      # and a more popular option exists.
    
      >You have installed the current version of mitsuhiko.flask, however
      >mitsuhiko.flask has not been updated since 7 months and there is a more
      >popular package available of the same name from another author.
      >
      >*****************************************************************************
      >Attention: pip does not check if the package actually does the same thing, be
      >sure to check that before you switch to a package from another author.
      >*****************************************************************************
      >
      >forker.flask (5000 installations)
      >mitsuhiko.flask (4000 installations)
    
      pip install -U forker.flask
    
      >Downloading/unpacking mitshuko.flask
      >…

~~~
hannesm
A name, once taken, cannot be unclaimed. If the package should be removed, an
empty delegation will be put there.

