
Short-Lived Certificates at Netflix - prabaths
https://medium.facilelogin.com/short-lived-certificates-netflix-fd5f3ae5bc9
======
viraptor
This is also implemented in OpenStack / for HPCloud in Anchor:
[https://wiki.openstack.org/wiki/Security/Projects/Anchor](https://wiki.openstack.org/wiki/Security/Projects/Anchor)

You can use it as a standalone project in your environment as well. There's a
talk about it at [https://youtu.be/Q_ZhrQq-_YM](https://youtu.be/Q_ZhrQq-_YM)
(ideas very similar to Netflix)

------
gruez
dumb question but, but why was oscp stapling invented when it's the same as
short lived certificates? have the certificate's expiration date set to a
short period, have the CA renew it regularly, and place it on some http
server. then you can have some cron job that downloads the certificate and
reloads your server. and since the certificates are short lived, the
CA/browser vendors can mark them as being excluded from OSCP checks. all the
benefits of OSCP stapling, without the extra implementation complexity.

~~~
merb
certificates costs money and the world was mostly manual in the past (and in a
lot of places still is) you can still order a 3 year cert.

~~~
yeukhon
Also, many CA don’t provide programmatic way to order and to download cert.
Another annoying manual step, especially consider having multi-domains not
convered under a single wildcard.

Sadly let’s encrypt does not support wildcard... and support for intranet is
controversial to thosr who want to keep their intranet “totally private”. I
love the autobot. One command to generate new cert, another command to reload
my Nginx.

~~~
JetSpiegel
Let's Encrypt will support wildcards in January 2018.

~~~
CaliforniaKarl
The issue we've run into with Let's Encrypt, is that they have a limit on the
number of new (non-renewal) certs in a time period, grouped by high-level
domain. So, for example, when you have lots of separate groups running sites
until a common domain (groupa.example.com, groupb.example.com), you often hit
the new-issuance limit, and have to wait.

So, I expect existing CAs (and groups like InCommon) will continue to be
around to serve large entities.

~~~
majewsky
Wildcard certs should help with that. Also, the limit is only on certs, not on
domains. If your process allows for issuing a single cert for 100 domains,
that also solves the problem. (There is a limit on SANs per cert AFAIK, but I
don't have the exact number.)

~~~
viraptor
If you're dealing with a large scale deployment that involves many endpoints
(and not just subdomain-per-user scenario), there are good reasons not to use
a wildcard. Other CAs will definitely stay in business there.

~~~
gregmac
> there are good reasons not to use a wildcard.

Such as?

~~~
viraptor
Example: Imagine Microsoft giving anyone "(wildcard).microsoft.com" just
because they have a lot of services to deploy and don't want to deal with
separate certs. Now, breaking into that service means you can mitm windows
updates for anyone and they can't tell a difference.

You want to limit the exposure of your certificates, you don't want 50+ teams
to share credentials/certs (they're effectively public at that point), and you
want to make sure that if you need to revoke the certificate _right now_ , you
know you're impacting as few endpoints as possible.

~~~
lucaspiller
It is possible to get a certificate that allows to sign other certificates on
your domain? For example you call Verisign and they issue a cert for
*.mycompany.com, then you use that to sign another certificate for
accounting.mycompany.com?

~~~
viraptor
Yes and no. If your company is worth a few million dollars, can get high
enough insurance, can get FIPS certified, and can jump through enough hoops,
then yes - you can get an intermediate certificate. That product is not even
openly advertised by most CAs. If you're just a "normal company", no, you
won't get one.

[https://translate.googleusercontent.com/translate_c?depth=1&...](https://translate.googleusercontent.com/translate_c?depth=1&nv=1&rurl=translate.google.com.au&sl=auto&sp=nmt4&tl=en&u=https://www.geotrust.com/de/enterprise-
ssl-certificates/georoot/&usg=ALkJrhjsNcK-HRUEkR4Z8w32xMoO7g19EA)

The big problem here is that the intermediate CA isn't really limited on what
they can grant. You can issue a valid "google.com" cert just as well as
"foo.yourcompany.com". The are x509 extensions for limiting the scope in this
case, but I don't believe they're widely used or validated at the moment.

------
nimbius
Does anyone know if ephemeral/automated cert issuing and renewal exists as an
open source project yet? Most of this is Netflix internal but I feel like
letsencrypt has made short lived certs an inevitibility

~~~
SoMuchToGrok
Somewhat related: Vault has a PKI backend that can help facilitate this.
You'll need to create some tooling around it, but we've had great success
rolling it out at my company.

[https://vaultproject.io](https://vaultproject.io)

------
CaliforniaKarl
I am looking forward to ACME 2.0 becoming an RFC! Once that happens, I can
ping our ACS team to start bugging InCommon to spin up the appropriate server
components. My guess is the other CAs are waiting for ACME 2.0 before they
really spin up support for it.

For anyone interested in tracking the progress of ACME 2.0, take a look here:
[https://datatracker.ietf.org/doc/draft-ietf-acme-
acme/](https://datatracker.ietf.org/doc/draft-ietf-acme-acme/)

~~~
majewsky
I don't know much about ACME 2.0 (except that it is apparently necessary for
LE to be able to start offering wildcard certs). Can you expand on why CAs are
waiting for 2.0?

~~~
mholt
Probably because it's the first IETF-approved spec for the protocol; right now
it's all in draft status. LE itself will be adding ACMEv2 next year.

------
tzs
Why is certificate management not integrated with DNS? You already have to
consult DNS to get an address to connect to, so why not piggyback certificate
validity information on top of that? I'd suggest allowing both revocation
lists and a way to say that only a specific list of certificates is allowed.

~~~
sytse
Great idea, I think that is done in [https://en.wikipedia.org/wiki/DNS-
based_Authentication_of_Na...](https://en.wikipedia.org/wiki/DNS-
based_Authentication_of_Named_Entities)

------
scurvy
How does Netflix handle applications that don't have certificate/key hot
reload capability? MySQL is especially guilty of this. It's a PITA to force a
restart just to reload certs or keys even once a year. I can't imagine having
to do this every few days.

------
moulidorai
Companies should start using tools like ManageEngine Key Manage Plus
[https://www.manageengine.com/key-manager/](https://www.manageengine.com/key-
manager/) (or) other similar products for secure ssh key and ssl certificate
management. Automation is the only way to avoid security issues.

Disclaimer: *I work for ManageEngine.

~~~
rgooch
Symantec has developed an Open Source system for short-term SSH and SSL
certificate management with 2FA (VIP and U2F). We encourage people to adopt
this to improve their security. Code:
[https://github.com/Symantec/keymaster](https://github.com/Symantec/keymaster)
Design document:
[https://docs.google.com/document/d/1AW3UROCJqTc3R4MLJXxmPUNS...](https://docs.google.com/document/d/1AW3UROCJqTc3R4MLJXxmPUNS0OFNcsiQJ_Q4j
--5tQE/pub)

------
je42
the article mentions, that HAProxy cannot do a reload without downtime ? is
that correct ? no way around that ?

~~~
detaro
Was true for a long time, and still is for the current stable version. This
article goes in more than enough detail over the history and the various
workarounds: [https://www.haproxy.com/blog/truly-seamless-reloads-with-
hap...](https://www.haproxy.com/blog/truly-seamless-reloads-with-haproxy-no-
more-hacks/)

------
JoeyPardella
and short-lived tenures ;)

