Hacker News new | comments | show | ask | jobs | submit login
Milestone: 100M Certificates Issued (letsencrypt.org)
581 points by okket 83 days ago | hide | past | web | 184 comments | favorite

SSL certificate from a traditional provider valid for a year: $10.

SSL certificate from a traditional provider valid for two years: $20.

Automated SSL certificate generation and deployment via LetsEncrypt with zero human intervention and more importantly zero human intervention to renew it going forward - priceless.


That's the real value for me. At $10/cert, that's not even a rounding error. But manually generating a new CSR, uploading it via crappy web form, waiting a random amount of time, proving domain ownership by responding to an email (sent in plaintext), waiting a different random amount of time, downloading the new cert (again usually sent via plaintext email), and finally copying it over the old cert and reloading the SSL conf ... now that costs some serious time and time is money.

It's even more painful when you're a web agency with lots of clients. Especially if they have hosting elsewhere and you don't have access to their emails. You have to contact your client and either get them to set up a specific email forwarder or inbox for you to access.

You then have to wait for them to respond back to you and you'll mostly get a reply with something like "I thought our emails were with you" or "We don't know who our provider is". Like, you're paying someone for some service, how can you not know who is providing you that service?

Then when they do finally set the email up, you have to go through the process which you just spoke about. It's such a pain, but with LE, it cuts all of this crap out and makes for happier businesses.

> Automated SSL certificate generation and deployment via LetsEncrypt with zero human intervention and more importantly zero human intervention to renew it going forward - priceless.

That also means one can make effectual suggestions to webmasters.

I told a webmaster who maintains a forum about LetsEncrypt a few months ago and that I'd help them set it up just so that users aren't practicing the bad habit of logging in over http. I just went back to check and the forum is now https by default using LetsEncrypt certificate.

I couldn't have achieved that a few years ago. The pain of setting up SSL would have inescapably digressed into a low-quality flamewar about how PKI is broken.

The LE protocol is fully open and even specified in RFCs - I wonder if any paid cert providers will ever move to that type of painless automated renewal (plus automated billing, of course). This would allow folks to potentially get their "advanced" certs (EV, wildcard, crossdomain, etc.) renewed without the rigmarole that you mention (although some cert categories come with their own non-automatable processes).

Right now, I have LE certs for my main website and for a test webserver hosted on my laptop, both renewed automatically and painlessly through cron jobs. The fire and forget nature of LE is very convenient.

EV certificates are not allowed to be renewed automatically AFAIK.

The situation is that facts about the Subject recorded in a certificate must be Validated.

For the Domain Name Validation (which is the main thing web browsers and several other kinds of software care about) this is done by one of the 10 Blessed Methods. ACME automates three of those ten methods. Some others can't be automated (e.g. get a bona fide lawyer to sign a physical paper document saying they've checked you really own the DNS name)

EV requires also validating the Subject entity's name and place of business. This cannot be entirely automated today. But some CA issuers specialising in EV get very close, like an hour turnaround in most cases for businesses in places where they operate.

The intent in ACME is that semi-automated processes would be handled to one side of ACME, you'd do them offline, and you'd gain "authz" which are how ACME tracks validation, for the things which had been validated offline.

Also, it's not necessary to re-validate everything for every new issuance. Suppose you prove you're really EXA Metal Pole Location Executives, business #123123123 of Some City, Texas. A CA would be entitled to rely on having validated that for some time, I think it's currently up to three years. In that time you could issue and re-issue hundreds, even thousands of certificates with no extra fuss.

That's not a part of the ACME protocol (which is what Let's Encrypt implements), it's a policy decision of Let's Encrypt. Grandparent's comment is about other CAs offering EV certs over ACME.


EV guidelines have nothing to do with Let's Encrypt or ACME. They state that even for renewal all checks have to be done again, which cannot be automated.

Maybe not fully automated, but at least the provider could "start" the process, only requiring the company they are validating to answer some questions, reply to mail, reply to whatever kind of extra authentication/verification is needed, then the technical portion can happen automatically after that.

Yes, several CAs have indicated that they are at the least watching the ACME development, and it was intended from the outset that ACME would have other users beyond Let's Encrypt.

Also keep in mind that if you're a big customer the big CAs already had products where things are streamlined for you, so the annoying experience you get as a small operator is not necessarily happening at scale for a large corporation like, say, Disney.

What's the justification of traditional providers to charge for the certificates? Do they offer services or extended certificates that "Let's encrypt" doesn't or is it just a matter of "no one tried to offer it for free before"?

Currently their only upside is that they provide wildcard (and EV) certificates. EV certs can't be automated for obvious reasons, but LE doesn't support wildcard certs because they don't believe there's a secure way of providing an automated way of getting them.

Given that traditional CA models don't actually have much more security than LE (in fact from personal experience they're far less secure), I wonder whether they should be in the business of providing wildcard certificates at all.

One annoying thing I've seen less technical stakeholders ask about with letsencrypt is "why the padlock isn't green" in Safari/Edge. Unlike Chrome and I think Firefox, those browsers only display a green padlock for SSL certs with EV, which of course letsencrypt doesn't (and arguably can't really) do.

The "grey padlock" for an otherwise perfectly valid certificate seems to confuse a very small subset of users, especially when there's a ton of badly written articles out there advising users only to trust "green padlocks", given the behavior in almost all other browsers. Thankfully showing google.com not using an EV certificate is usually easiest way to shut these conversations down.

>> only to trust "green padlocks"

It's not so much about browser-specific behavior, it's about using the padlock to ensure that the company or other entity you're interacting with is the one you wanted, as opposed to only ensuring that the server you're talking to is the one indicated in the address bar.

A gray padlock is better than none, but telling less technically-inclined, clicks-all-the-links, fat-fingered users to trust them outright may not be the best idea now that you can go get bankofomerica.com, dress it up to look like BoA's site, and put a gray padlock on it for free.

Given how piecemeal the use of EV certs is by big companies, I'm not sure EV as implemented today delivers any 'real' benefits. It absolutely is about browser specific behaviors. How can you tell users only to interact with companies if they have an EV certificate, when huge internet properties like Amazon, Google and Facebook don't use them? In reality you can't, which means Safari/Edge's behavior arguably doesn't really help anyone. I'm not saying Safari/Edge shouldn't tell the user an EV certificate is present, they absolutely should, but that the approaches used by rival browsers arguably make more sense.

I think Chrome/Firefox got this one right by using a green padlock for both valid EV _and_ DV certificates like Letsencrypt, displaying the extra EV information clearly inline in the URL bar if it is present. The changing color in Safari/Edge can give the impression that the the non-EV certificate is somehow less secure, which in technical terms is not the case. Questions of identity verification and security of data transmission are two separate issues which the Safari/Edge approach arguably conflates - both EV and DV can provide equal security of transmission, even if only EV provides some degree of identity confirmation.

As a user, it's simply not possible to make the distinction too - if I visit your fake Bank Of America site with a grey padlock, how can I even be sure the 'real' site had an EV certificate and a green one, when there's so many companies that don't? As a user you are effectively forced to treat the grey and green padlocks the same, so why not make them the same color and display extra EV info if it is there, like the other browsers?

I agree with you. My point - which I admittedly didn't state very well - is that the best-practice messaging to non-technical users needs an update. In the earlier days of the web, it was as simple as "look for the padlock," because the state of SSL was such that generally only legit sites had it. With the presence of LE and the overall failure of the "web of trust", this is no longer good advice - they need to carefully double-check the address bar as well.

What types of users have asked you about that? I'm lucky if I can get regular users not to click their way through cert errors.

Like you I've never seen a "customer" user notice, which is why I was careful to use the word "stakeholder". For me it's almost always been product management/sales types in the company.

Why is providing wildcard certs not secure???? Amazon seems to do it?

Because once your wildcard certificate is compromised--and you must design systems with the assumption that your certificates will be compromised--it becomes a very achievable exercise to use that wildcard certificate to compromise users going to any of your subdomains. With each domain governed under its own TLS certificate, you install firebreaks between systems to contain the damage an attacker can do.

There are vanishingly few reasons to ever share private keys across multiple automated systems. This is not one of them. (Are your systems not automated? Fix that.)

So the Low risk of compromise for most people is more important than the ability to easily maintain a system? If a certificate is going to be compromised it's most likely going to be the most important cert to the system anyway. A hacker isn't going to go after the meaningless ones. But by your logic I should increase the maintainability of the system for no benefit.

Obviously this is highly dependent on the usage of the certs to begin with.

Meanwhile SAN is fine? The alternative to wildcards is that you get a single cert with tons of subdomains as alternative name. Exact same risks, yet much more hassle to add subdomains. I don't see it.

That's a fine point, if you're using SANs. With Let's Encrypt, you don't need to use a SAN so long as each of your endpoints is distinct. (If they're not, I'm uneasy with your architecture in the first place, y'know?)

However, I would also point out that wildcards allow people who aren't you to create domains that you didn't create and represent them as being from you, which is an attack vector that even SAN enumeration doesn't allow. We're talking whitelists over blacklists.

SNI is not yet always an option if you have to support older clients.

It's not about if I'm using SANs anyway. It's that LE is fine with that, but not with wildcard certs.

> However, I would also point out that wildcards allow people who aren't you to create domains that you didn't create and represent them as being from you, which is an attack vector that even SAN enumeration doesn't allow. We're talking whitelists over blacklists.

Can you elaborate what you mean here?

If you have to support older clients--definitely, SAN is your only option if you have to run on the same host. (And I sympathize, because that sucks.) At that point, though, my inclination becomes to attach a bunch more public IPs and wire up my DNS correctly to match.

> Can you elaborate what you mean here?

Me saying "this cert is valid for example.com, foo.example.com, and bar.example.com" is an incrementally better security posture than saying "this cert is valid for *.example.com". It constrains an attacker from running a phishing attack from, say, "private.example.com" or "secure.example.com". You're whitelisting what you're covering by explicitly declaring all of them.

It is incremental, but it's a thing.

IPv4 addresses are relatively expensive. I encounter it quite often where a wildcard gets pointed to a single server (with single IP) which hosts multiple services. There are no security issues compared to a large SAN that I can see in such a situation.

I would submit, respectfully, that you're not being creative enough.

When it comes to security, including phishing, it behooves all of us to not assume that permissiveness is a de facto positive.

Very helpful. Thanks.

Yes, many services "seem to do it". The problem is how do you safely verify that a wildcard certificate is actually being given to the owner of that domain space? It's "trivial" to provide a certificate for a particular domain is owned by someone, but the same techniques don't work for wildcard certificates because now you're verifying only a single domain and providing certificates for an entire class of domains.

If you work through all of the ways ACME verification works (/.well-known/, DNS, and so on) you can come up with several plausible setups that become insecure if you allow wildcard certificates.

Here's a stack exchange question about it: https://security.stackexchange.com/questions/157972/why-cant....

That SO post makes no sense to me. It claims:

> The authentication test can't merely back down one level to mysub.example.com because that isn't covered by the cert requested, and may not be under the control of the requesting agent either.

This just seems wrong. Performing the check like that seems to be a sane way to do it. If people can't control the base domain they have no business requesting a wildcard cert.

But *.a.com doesn't cover a.com, so it's not very clear that the inverse ownership relationship should hold. Yes, it's true on a DNS level but that's one argument I would imagine against doing it that way.

No, but ownership of a.com heavily implies ownership of (wildcard).a.com (HN is being weird with the asterisk)

Alternatively they could just do a DNS check. If you can configure DNS for a.com, you control *.a.com.

> heavily implies ownership

"heavily implies" is too low of a bar to make it safe for a CA to provide wildcard certificates through an entirely automated process. While they could do it, they care enough about security to make the decision not to.

> If you can configure DNS for a.com, you control *.a.com.

This assumes that if you can create a TXT record for a.com that you must also have access to the ability to create other records for a.com.

While it is a stretch, I would say that the only bar sufficiently high would be fulfilled is forcing an administrator to prove that they can change the NS records for a domain. Which I don't believe would be a practical thing to test, and might run into problems with things like cloudflare.

> This assumes that if you can create a TXT record for a.com that you must also have access to the ability to create other records for a.com.

Yup! And LE is totally fine with this! Have you read the ACME DNS verification spec?

I regularly have subdomain zones delegated to AWS for Route 53 use without having direct control over the root domain. The idea that I must have direct control to assert control over a subdomain is pretty questionable.

If you don't have access, fine. You can't make use of this functionality. Stick with what is available today. Nothing lost. If you do have access, like 99% of the domains out there, you now get access to more functionality. Win-win.

Just making a certificate costs almost nothing. So in a sense $0 seems like the right price.

BUT, being _able_ to make certificates is very expensive. ISRG (the charity behind Let's Encrypt) spends _millions of dollars per year_ on this.

Purely mechanically Let's Encrypt needs a data centre with a bunch of high availability servers and pretty good Internet access, employees to look after all that stuff, a deal with CDNs to serve up their OCSP responses around the world (which ensure certificates stay working after they're issued) and so on, none of which is cheap and all of which needed to be in place before issuing even one certificate.

In practice to be any use the certificates need to be trusted, which meant meeting the standards of the big Trust Stores (definitely Apple, Microsoft, Mozilla, probably Google, Oracle, and ideally dozens more besides). Those standards require physical site security, regular third party audits, and so on, which cost even _more_.

Now, for Let's Encrypt the solution is this is a charity, you can pitch in $10 and if a hundreds of thousands of people like you do that then it all gets paid for, or if a few bigger organisations pitch in $1000 each that helps too.

But for a commercial outfit, well, why not charge $10 to cover costs? And if $10, why not $50? Why not $1000?

> What's the justification of traditional providers to charge for the certificates

They could, so they did.

Among other things they provide certs which are valid up to three years. Let's Encrypt certs are short term. There idea is to auto renew, but sometimes that is difficult to implement.

No-one did free which works in all browsers before.

StartSSL worked with any Windows XP SP2+ computer. Though you couldn't automate it and their website was kinda slow.

And they demanded personal data. Even checked that your personal address is not a business address! "Domain validation" my ass.

Is this total BS then?:

"Why pay for something that’s free, you ask? When a user visits a website that is secured with a self-signed, or free SSL certificate, most web browsers will post an error message. While some people will click “I understand the risks” and proceed to your site, it’s likely that many people will click the “Get me out of here!” button and never return.

The real problem lies in the fact that self-signed certificates are virtually unregulated. If your site is compromised, it may still appear secure; however, certificates issued by a trusted certificate authority can be revoked and therefore alert users of any potential threats."

Source: http://searchengineland.com/comprehensive-guide-ssl-certific...

Sort of, that's conflating all free SSL certs with self signed certs. The whole point of an SSL cert is to provide authentication that the server you're talking to is really the server you were trying to reach at www.example.com . They are talking about self signed certificates which is basically the remote server saying "I'm www.example.com, you can trust me because I said so". Let's Encrypt is the same as other CAs out there in terms of authentication. In fact, if you see a certificate that's signed by Symantec, it's more likely to be mississued than a certificate signed by Let's Encrypt.

The certificates Let's Encrypt issues are _not_ self-signed. They are signed by a trusted certificate authority which Let's Encrypt manages, and are no different from domain validated certs signed by any other CA.

That article makes the mistake of assuming that if a certificate is free then it _must_ be self-signed, which is obviously false.

The price is nice, but what makes it a must have for me is the ability to generate a cert with 100 domains on it and have it validated by hitting the server.

The "old" way required all customers/domain owners to either give us control of their domain or to answer an email every time we needed to get a new cert. Now we've automated moving around domains between certs and generate new ones as needed. Validation by email to 100 different people is like herding cats. Validation that happens in an automated way - priceless.

With SNI, what's the advantage of having a cert with 100 domains on it ?

Also, HTTP/2 can reuse the same TCP connection for multiple domains with SAN

With our setup, certificates are handled by Amazon elastic load balancers. A single ELB with a cert for 100 domains using SAN is easier to manage than 100 certs on a few servers using SNI.

I'm not saying you should design this way from the start, but this is what we grew into and SAN works for us. I only wish the limit of 100 per cert was higher.

Older clients. Windows XP, android 2, older java versions

> At $10/cert, that's not even a rounding error

For businesses, sure, but the fact that you had to pay money at all was very discouraging for personal websites!

Totally agree. Actually wished there was a paid acme service, basically the same as letsencrypt but without such low rate limits.

And this has been the same problem with pgp and crypt ion in general. Too costly in man time. To much to manage, regenerate, etc. unless it becomes transparent or setup once and forget it encryption will never get easier to deploy or manage or be more widely adopted.

> Automated SSL certificate generation and deployment via LetsEncrypt with zero human intervention

Unless you are on a shared hosting, which is still one of the most popular options.

This is an interesting situation where "public good" happened to align well with business goals of some deep pockets.

Particularly, Google and Akamai...two of the biggest LE sponsors. They both retain good visibility to user behavior (like specific urls visited) because of things like GA,MITM proxying, etc. But, ubiquitous availability of that is taken away from ISP operators.

Which is a good thing. Makes me curious if there's anything else like this that could be achieved. Are there other net public good projects that align well with deep pocket potential sponsors?

What also may have motivated Google & Co was that ISPs and, further down, hot spots like hotels etc. started to insert/replace ads on unencrypted connections...

Security professionals tend to be motivated to work on improving security because they think it's important to protect users from the bad guys. And it's not like people have a sudden change of heart when joining Google.

But it is also not like you get to do what you want in big corporations. Rather you do what you have been told, and usually you might not even know the real reason.

Also security professional is a person that know a lot about security, his intentions are not necessarily "good" for him to be considered a security professional in my opinion.

Very good black hat hacker are also professionals.

That's rather more top-down than I've seen. There are large corporate initiatives, but many projects happen because someone thought it was a good idea and sold it to management.

For plenty of people, Google is a bad guy.

But there needs to be investment to pay said security professionals to develop the tools, publish them, polish them to be user-friendly (otherwise no-one uses them, see GPG), to pay for the datacenter hosting...

It doesn't happen in a vacuum.

I think they nail their point with "it illustrates the strong demand for our services."

Letsencrypt is cheap (free) and easy to use.

Even people with not a lot experience can secure their sites and apps, and it just works. Yes, you have to update it every three months, but that's worth the price and the excellent documentation.

Before letsencrypt I always wanted to secure my blog with https but never got around to it or it just looked super complicated and error prone.

Then, my provider built its own tool for LE and its just so easy to implement.

> Yes, you have to update it every three months, but that's worth the price and the excellent documentation.

With certbot it's actually easier to update every three months with Letsencrypt that updating every/other year with a traditional SSL CA.

> Even people with not a lot experience can secure their sites and apps, and it just works. Yes, you have to update it every three months, but that's worth the price and the excellent documentation.

This is just a cronjob, no?

Cronjob with a script that needs to do some back-and-forth/handshaking, but yes, it is.

Yes and no. I ran into an instance where certbot itself needed to be updated and the scheduled renewal didn't take place until I did so the next day.

I also use Subsonic @home which requires generating a keystore as the app doesn't yet handle LE natively.

If you're comfortable with said cronjob having access to your private key

If the private key is only used for this purpose, and the cronjob as well as the key only reside on the server, is the security of the key then not a moot point if the server is breached? Genuinely curious.

I don't store keys on the filesystem for two reasons:

* Security in case of a breach

* Ephemeral containers

I try never to store secrets in an unencrypted cloud filesystem, and decrypted secrets should be in memory, if possible.

I store my LE keys in Vault using ten-ply-crest, so access is very restricted yet storage has redundancy. Vault is dead easy to integrate with any system.

I have been really surprised how many LetsEncrypt tools store certs and keys in the filesystem, other than "that's the way it's always been done".

Also if the keys are stored only on one cloud server, that means there ought to be a backup somewhere, so now there are two potential leaks instead of one.

With Vault only ten-ply-crest and Fabio access the keys. No human ever gets near them.

If you update the certificates manually, then the private key would just reside on your local machine and not need to be exposed on the server.

If the server requesting the certificate is going to use the certificate, then the server ends up with the private key either way.

Indeed, but I cannot see how this addresses my question?

because your private key never needs to be compromised, even if your server is breached, because it never needs to be on the server. That said, I agree that if the PK is only used for this single server, you have bigger problems than its loss if your server is breached.

If the key isn't on the server how does software use the certificate?

ACME doesn't require that the client requesting issuance has access to the private key of the certificate. Access to a CSR (which only includes the public key) would be sufficient. Most clients support a mode where they use existing CSRs rather than requiring a private key, e.g. certbot with --csr.

I now just run all of my sites behind Caddy which takes care of cert renewals for me. Can't recommend it enough!

Until LE goes down and you can't restart your sites.


> Yes, you have to update it every three months

I don't have to update it at all. A cronjob does that for me now.

I've been very happy with certbot-- the LetsEncrypt automation tool-- and can attest to the ease of use.

> Yes, you have to update it every three months

It can be automated. On mine, cronjob does the job!

acme.sh reduces SSL deployment to fifteen minutes with virtually no maintenance.

I still wonder how Let's Encrypt works.

I understand that the problem they solve: A user wants to get the public key for a certain domain. So he knows he is talking to a server by the domain owner and not some man in the middle.

So he asks a third party whos public key he already has. In this case Let's Encrypt. Ok.

But how did Let's Encrypt get the public key from the domain owner?

I know they make the domain owner install some software on his server. But how does that make sure they talk to a server by the domain owner and not a man in the middle? Does the software include some private key so they can send them a "hey, encrypt this" message and to prove the server in fact has that private key?

If so, why is the Let's Encrypt software so complicated and not just a 5 line script or something?

They have a variety of "challenges" used to prove control of the domain. They basically boil down to:

- LE makes a request to example.com, to a known URL that contains a key that LE told you to put there

- LE does a DNS lookup for a specific TXT record (again, containing a key they told you to put there).

The complete answer to your question can be found in the ACME spec (the protocol LE uses):


LE probably does that multiple times from a few places around the internet, to reduce the likelihood of something unexpected going on.

Ain't nobody got time for that!

They might have "a few places around the internet" to verify from, but I doubt it is very diverse location-wise. And there's no way they're going to bother with doing it multiple times, for free.

> they're going to bother with doing it multiple times, for free

You are aware you are talking about the cost of a single HTTP GET?

> You are aware you are talking about the cost of a single HTTP GET?

First, the conversation is about the possibility of multiple (not single) interactions, including protocols other than HTTP such as DNS.

Perhaps you're confusing a subset of what they currently do vs. coordinating the initiation of requests multiple times from a few places around the internet (per OP) supporting multiple protocols, aggregating and verifying the responses, etc. -- all within a reasonable timeframe at scale with Internet infrastructure level high availability (while protecting against malicious misuse)?

I hope this is enough to explain that no, that's not what I was talking about. But thanks for taking the time to reply!

There is a nicely done 'how it works' on Let's Encrypt: https://letsencrypt.org/how-it-works/

I have read that, but it seems overly complicated. Why not just give the domain owner a private key and a script that is a few lines long. So he can sign a message from them and prove he has the key?

In the process you describe, there is no step where you confirm that you are actually the domain owner. Anyone could obtain a certificate for google.com.

Verifying domain ownership is what the challenges are responsible for.

In effect that is sort of what they are doing, but simpler.

You claim to have domain x.com. LetsEncrypt sends you a token. Place it in a DNS record or in a pre-determined path on your domain and ask LetsEncrypt to read it back.

If it matches, it is verified that you own the domain.

How is the scheme you propose simpler, whilst retaining the same base guarantee (that the requester is the owner of the domain in question)?

> in a pre-determined path on your domain

That would be easy and elegant. But they want you to install software on your server.

They want to make it easier, and that's the easiest way, but it's not required. You can perfectly well run it on your machine and copy the files manually to the server. You can even use a simple webpage as your local client: https://gethttpsforfree.com/

The only software you need is something that can speak the ACME protocol, which is what letsencrypt uses to receive your CSR (which you can generate yourself with openssl), issue challenge tokens and then (subject to you successfully passing the challenge) give you a certificate.

You don't necessarily have to do any of the above on the server for which you want the certificate. You can do it somewhere else and then transfer the resulting certificate to the intended host (either manually or with a shell script/ansible/puppet whatever).

The software just negociates the token and such things, and communicates with EL to say “I want to have a certificate for such and such domains”.

But you can place these tokens on your webserver by hand if you want, in a pre-determined path.

How do we know that the recipient of the key in your scenario (or the originator of the CSR) is in fact the domain owner? The domain contains contact information, but the exchange can't be done by email - that's not secure. Can't be done by SMS - that's not secure. We don't know them personally, so there's no obvious out-of-band technique that can be used.

We'd have to go back to physical snail mail using the address on the domain record, and/or notarization. Both of which of non-automated and very slow (relatively speaking).

I quite like using DNS verification for this sort of thing.

I always balked at the hoops I had to jump through until Let’s Encrypt came along. I now have two boxes, one Gentoo, one CentOS updating every three months pretty much pain-free. I know it's not a valid counter-argument but I'd say that 100M certs issued and the general tone of user feedback says you're wrong about it being overly complicated.

To know you're talking to the domain owner, I think you'd have to use the contact info from the whois info. I don't believe that's especially consistent across tlds.

The http or dns challenge/response is more reliable and pretty easy to automate and scale to many domains.

There's two ways. You can include in DNS a TXT with the validation string they give you during install, meaning you can do it so it's yours. Or you can add a special file in a special directory of the website behind the domain you're asking a certificate for. So it means you've got access to it so you're the owner, or a technical third party who can do that.

Of course it doesn't prevent a total hack of your infrastructure but when you've come to that point, you've got bigger issues.

> A user wants to get the public key for a certain domain. So he knows he is talking to a server by the domain owner and not some man in the middle.

> So he asks a third party whos public key he already has. In this case Let's Encrypt. Ok.

It works quite differently -- the browser doesn't have to talk to Let's Encrypt at all to verify correctness.

How it works:

1. Browser connects to the server

2. TLS handshake, crypto-stuff

3. The server cryptographically presents the browser his certificate

4. The server's certificate is the server's public key, some attributes (like what domain names the cert is valid for) and a digital signature of all that stuff

5. The browser has a list of certificate authorities

6. The browser checks whether the digital signature on the server's certificate is valid and was made by a CA the browser trusts

7. The browser checks that the certificate is valid for the site it's currently accessing

8. Connection legit

The problem that the CAs (including Let's Encrypt) solve is not distribution of public keys, but rather trusting public keys presented by someone.

> But how did Let's Encrypt get the public key from the domain owner? > I know they make the domain owner install some software on his server

What the ACME clients do is two-fold:

1. They put up some sort of challenge response mechanism. This is used by Let's Encrypt to test whether a client is authorized to rule over a domain.

2. They generate (or use) a "Certificate Signing Request" (CSR). A CSR is basically a digital "sign here" field. The CSR contains the server's public key, what domains it wants to represent etc.. The CA checks the CSR using the challenge-response mechanism from (1) and then signs it digitally. The result is a certificate, which is sent back to the client and installed by the client. The private key of the certificate never left the server.

> If so, why is the Let's Encrypt software so complicated and not just a 5 line script or something?

Well it doesn't have to be complicated. acme.sh is only five (thousand) lines of shell :)

There are a bunch of ways the challenge-response mechanism can work, and there are also a lot of other options, which add to the complexity, as well as installation routines for certificates, which support a bunch of different servers.

A specific case with "static" challenge-response could conceivably boil down to just a dozen lines of shell or so.

Nit pick: Although toy systems work by "just" signing the CSR a real CA like Let's Encrypt basically throws the CSR away after verifying its integrity.

The CSR is a signed data structure that says "Hi, here's my public key K1, and I want you to issue me a certificate with properties P1, P2, P3 for names N1, N2, N3" and it's signed using a private key K2.

Using K1 (which is right there in the document) we can check that the K2 signature matches. If not, we can't be sure the real owner of K2 even wanted this certificate and we must reject it.

Then we can look at the list of P1... and N1... and decide if we want to issue a certificate at all. If we do, we can choose to issue any certificate we want, we don't need to issue exactly the one described in the CSR.

This is important because many CSRs, whether generated by well-meaning admins or automatically by an appliance or service, are full of crap that Let's Encrypt doesn't want to, or isn't permitted by the Baseline Requirements to sign.

[ e.g. it's very common for CSRs to claim that London is in a County called Middlesex. Nope, that county was abolished decades ago, London isn't in a county at all, but people see a field labelled "State or County" and they figure they should write something in the box. If a CA signs that, they're claiming Middlesex still exists. ]

By constructing each certificate from first principles after examining the CSR, Let's Encrypt can ensure they sign only certificates for exactly the things they've validated, nothing else.

Thank you, that's a good and important point. It also has the angle that the CA can't accidentally "miss" anything, since the certificate is not constructed from client-supplied data (=> voila, we're at Langsec :)

The biggest issue we've had is the short expirations. We have 51 certificates in our organization and do not want to rely on auto-renew.

As our community project, we built a totally free to the public service to monitor certs and alert you when they get close to expiration or are invalid, etc:


Feel free to use it for any certs.

Out of interest, what makes you not want to rely on auto-renew?

It has never worked right on Windows servers (don't know why). And it is simply a check of last resort if something goes wrong.

LE already sends me notifications by email. What am I missing?

Let's Encrypt's renewal notifications don't cover cases where a new certificate was issued, but for some reason isn't being used by a server, perhaps because the ACME client ran into a problem, or because the server wasn't reloaded. It's useful, but it doesn't quite replace actual certificate expiration monitoring.

This smells like a problem in your server configuration management tooling, yeah?

Not monitoring something you rely on - like the expiration date of your certificate - would be the problem. Assuming that the entire process is bug-free sounds like a bit of a gamble.

If your CM tooling is working correctly (because you wrote tests, yeah?), it should scream and fail in this situation. "I should never have a cert that's out of date."

Again, you're assuming whatever automation you built is bug-free. Practice shows that sufficiently-complex software is never bug-free, no matter how many tests you wrote. Checking the expiration date on a certificate is easy, so why not have that safety net?

If I'm worried about that, Chef is performing those tests.

I'm saying, you're putting it in the wrong place. It's not a monitoring problem, it's a configuration management problem--"I have defined a policy, am I in compliance with that policy?". And, from experience consulting with a pretty wide range of clients, putting it in the monitoring stack encourages manual work instead of automation. Instead of having active and self-correcting systems, you create passive and manually reactive ones. Monitoring is for stuff you have to react to. Automation and automation testing is for stuff you should never be reacting to.

If your CM is testing that the expiration date of the certificate that is actually used by a TLS server is correct, through whatever means, for example after each renewal operation, and if you also have some way to verify that the renewal operation runs regularly, then you're effectively monitoring your certificate expiration date. I'm not particularly concerned with where you do it from, whether it's part of your CM or a third-party service, etc., as long as it happens.

There are plenty of other things related to your TLS certificate you might want to monitor that don't quite fit into CM (OCSP revocation status and availability, unexpected key rotations, etc.), so you'll likely need separate monitoring either way.

That's a neat service. If anyone is looking for a self-hosted option, this looks promising: https://github.com/RaymiiOrg/certificate-expiry-monitor

Why would you not use the auto renew system?

Oh, and it also does some validity checking.

Thanks for providing that!

I've been using another free certificate monitor, but it assumes 1 year validity and the lowest possible warning threshold is 30 days, which isn't all that useful when working with Let's Encrypt.

So, you have another happy user now :-)

This one also works: https://sslping.com

A good service.

While registering, I noticed timezone as "Indian Time". It should be "Indian Standard Time(IST)". </nitpick>

Genuine question: Are the other smaller cert-issuing services going out of business? If not, what has been their response to LetsEncrypt?

Not that all of them should survive, there are a lot of crappy services that deserved this. But I'm just trying to place myself in their CEOs position and wondering how the game plan should be.

StartSSL effectively died, and I assume LetsEncrypt was part of the reason.

No... the real reason StartSSL died was because they were acquired by another CA, Wosign, which had a number of serious security and management issues. These issues led to both CAs being distrusted by all major browsers.


Right, but why did Eddy Nigg sell StartSSL to WoSign?

My guess is Heartbleed made him re-consider whether he wanted to keep doing ssl/tls certs, and once Let's Encrypt came along he started looking for a way to get out of it.

Go upmarket and sell EV certs or codesigning certificates.

There are some niches left even for DV like wildcard certs or longer validity periods.

Wasn't the Snowden releases part of the motivation for Let's Encrypt? If the web is "encryption for everything by default", then dragnet survellance from the NSA (etc) is much harder.

Shameless open-source plug: if you need a automatic LetsEncrypt module that works with your containerized environment, try ten-ply-crest [0]. Works best with Consul where it automatically registers based on service tag, and can securely store certs in Vault, which makes it a great match for Fabio [1].

This way you can run multiple load-balancers and keep your existing infrastructure while still enjoying all the benefits of LE. Written in JavaScript but works with any stack.

Based on the great rawacme, and we'll keep this updated as ACME hits 2.0! [2]

[0]: https://github.com/nextorigin/ten-ply-crest

[1]: https://github.com/nextorigin/ten-ply-crest/issues/27

[2]: https://github.com/AngryBytes/rawacme-node/issues/2

How timely. I got my first LE cert two days ago! I used caddy, a really nice http server and proxy that's HTTPS first [0].

[0] https://caddyserver.com

The graph of HTTPS percentage has some sudden massive dips a little bit ago. Does anyone know the reason for those?

I love letsencrypt but I wish they would hurry up with deterministic dns challenges. It would make securely automating certificate renewal vastly simpler and easier (especially for certs that are being used for non www serving endpoints e.g. mail servers etc).

I worry that at the moment there are probably a lot of systems out there that leave DNS API keys lying around on endpoints because it's easier to automate the renewal than write a convoluted two stage ansible role/play that delegates sensitive actions to the controller running the play.

This is due to the CA/B Forum Baseline Requirements stating that domain validation via DNS records (and pretty much all other methods) has to use either a random value or a single-use request token.

There's a neat trick with CNAMEs and a separate DNS zone that works kind of as a workaround, if you're okay with introducing another piece of infrastructure[1].

[1]: https://github.com/joohoi/acme-dns

Yeah I read somewhere that they were working to try and get the CA/B rules changed to allow it (no idea if this is true).

Will check out the CNAME trick - thanks!

LE really fills a big need for cert issuance for massive web hosts with custom domains (WPEngine, Hubspot, Shopify are all on LE) as it both greatly reduces the support burden of setting up certs for all of their sites and sidesteps some of the limitations of the legacy cert structure.

I've been using kube-lego (https://github.com/jetstack/kube-lego) to automate certs for Kubernetes ingress for the past 9 months. It is a joy to just add a domain to a manifest, kubectl apply and then hit a browser with a certificate working. Thank you LetsEncrypt for making the Internet more secure.

And most of that is because LE [still] doesn't issue wildcard certs, nor does it really plan to :(

And that is a without-exception good thing. Wildcard certificates are dangerous and encourage bad practices around SSL. You shouldn't use them. Instead, you should fix your workflow to know exactly what domains you have and have active and use automated tooling to provision them correctly.

I don't think so.

I have an industrial VPN with ~300 embedded devices behind them. For some reason the user wants to access to each one of it via https.

I thus set up an Apache proxy which proxies https://device_n.vpn.cust.tld to the VPN device (and some auth)

LE cant issue 300 certificates and it would be very ugly.

I don't believe LE has an upper limit on its number of SANs.

For systems not explicitly public, whitelists are almost always better than blacklists.

> I don't believe LE has an upper limit on its number of SANs.

The limit is 100, unless they removed that when I wasn't looking. Back in the beta I tried one domain, a handful and then, because I couldn't find a technical reason why SAN should be limited, I tried 500. It fails. I have many certs with up to 100 and would be happy if this changed.

The highest I've seen in the wild is 60 or so, so I'll bow to your superior getting-bitten-in-the-ass-by-it. =)

(And if so, in the prior commenter's situation--okay, create three certs. LE might not support 300 certs, but 3 with 100 SANs should work just fine.)

Currently security sandboxing happens basically per Origin. Since schema and port is fix, and I don't want to force every user/potential-user/demo-user/anon to register a domain (so the LE rate limit doesn't apply, and nobody want's bloated 100 SAN certs).

I know the names, but rerolling a cert takes time, takes up resources, etc.

I'd like to have the list with those 100 million websites, I didnt know there are so many.

It's not 100 million web sites, because Let's Encrypt certificates expire every 90 days, and you have to get a new certificate if you want to add or remove a domain name, and you can get duplicative certificates as long as they're within the rate limits.

As I said in my post at


it's probably between 17 million and 46 million web sites, although it could be a bit smaller than 17 million at the most restrictive notion of "web site" if you don't consider subdomains to be a separate web site even if they have different content and are operated by a different person.

You can see all of the certificates at

https://crt.sh/?Identity=%25&iCAID=7395 https://crt.sh/?Identity=%25&iCAID=16418

and you can also get a lot of the data from


or https://www.censys.io/ (registration required), although the people running it told me in connection with this that their Certificate Transparency data is not as current at the moment as crt.sh's data.

Shame they can't issue wildcard certs. I don't see why they cannot.

It was, and I think it still is, a very much requested feature. I for one will benefit from it in one particular setup. But so far, can live without it.

I would like to get a certificate with 3-years lifetime. I know that 90-days will limit the damage from key compromise, but I don't want to automate...

I was in the same boat, I didn't like having to run python on my server just to renew a SSL certificate. Still, a few months ago I decided to give it a try and I don't regret it. The certbot script is packaged in FreeBSD, it wasn't painful at all to setup and a crontask later it was over. Definitely beats the crappy free "startssl" certificates that I used previously.

There also are a lot of alternative [1] implementations which are less bloated, some are really small with only some hundred lines of code and few dependencies.

[1] https://letsencrypt.org/docs/client-options/

acme.sh runs in pure shell, no need to install Python. It's my goto client and it just works.. everywhere.


Seconded. I love that it supports custom port numbers, so I can use my not privileged web server port.

acme.sh + static verification is indeed a very nice zero maintenance solution.

If you use the DNS verification, you could run the client on your local machine and then upload the cert.

I started writing a small utility to do just that (and then upload the cert to CPanel using their API). It works, though I'm lazy to make it reliable enough for release.

> startssl

I must have at least 4 accounts there due to them using client certificate login. Great proof of concept why certificate based website logins never took off.

you're perfectly free to pay $10/year for a manually issued certificate from one of 50 other providers

And this is what I'm doing right now... :-)

Just hope your certificate provider doesn't screw up or you may be forced to replace early.


Why not? There are even docker containers that will do everything for you, all you have to do is tag your application containers:


I use Dokku (sort of like an open source Heroku) and the LetsEncrypt plugin. I can deploy a webapp with a git push and have it run over https with no work on my part. Works great.

LetsEncrypt could easily sell extended expirations out-of-band. I see it as nearly inevitable when they need (more) money.

It's unlikely. Google and Mozilla are pushing hard for short-lived certificates, and the CA/B Forum has already passed a ballot that limits certificate lifetimes to 27 months, starting with March 1, 2018. Google wants to see that brought down to 12 months and might enforce that in their own products if the CA/B Forum doesn't keep up.

Pay $10 to disable the cron job running yet another pile of 3rd-party code on a production server, even if it was (eventually) just for a year? I don't see it as a "never!".

The market is probably (a) people who want to support/donate to LetsEncrypt and (b) people who are ultra-paranoid / strongly prefer less moving parts in ops.

The one major downtime I'm aware of so far (a month ago) could also be a factor: Let's Encrypt is down | https://news.ycombinator.com/item?id=14374933

Why not? Automating it once will prevent you from scrambling in three years, which is exactly LetsEncrypt's point.

Nearly 20K of them for Paypal phishing sites and who knows how many for others. While a noble intention, one can't ignore the damage they've done.

That's .02% of all certs issued. A road might enable a bank robber to get away faster, but nobody is complaining that we shouldn't build roads because of it.

That's only Paypal :) There are hundreds more instances of abuse

And 99,900,000 instances of non-abuse. A free service gets abused, surprise surprise. As LE have said, it's not their problem to solve typo-squatters, or IP infringement.

I did some work in this space. The cert authorities were never really designed to be a backstop against phishers as that threat really rose up in prominence after they were setup. And it placed them in a weird gatekeeper role.

As designed, it was supposed to be that:

Domain Validation (DV) certs would indicate that your connection was secure.

Extended Validation (EV) certs would indicate that your connection was secure and you were connecting to the proper site.

In reality, the problems with phishing and mis-issued DV certs got so bad that that issuers like GeoTrust (owned by Symantec) began keyword checking DV certs for major company trademarks, phishing targets, etc. that depending on what it was would put your through varying degrees of scrutiny.


- Some used bookstore had a domain like "the-good-book.com" which got flagged for manual review b/c of the "book.com" string and so many "facebook.com" phishing attempts.

- Anytime people would request certs for BMW, HBO, would trigger requests for contact information within those organizations and often a phone call from GeoTrust

- It was basically impossible to get anything issued with the term "paypal", "bank" or "apple" in the domain.

Please note that these were all for run of the mill - $15 single domain, DV certs.

This was for a fairly high end service that required credit card accounts, etc and we still saw around 5% of requests get pulled (rightly or wrongly) into one of these manual review filters.

It was a fairly unsung and unappreciated part of the what the cert issuers were doing - but it absolutely 100% helped.

LetsEncrypt is forcing the Cert and Browser industry back to the original DV/EV split and it's unclear where we're going to land.

Right now even technical people _trust_ the green lock and it's wildly easy to miss the fact that you're on a domain that is paypal-login-security-usa-1.com

I think we're going to see a future where the browser indicators are:

Red -> No SSL

Gray -> DV SSL

Green -> EV SSL

Browsers should also completely drop that lock icon for any type of connection as it's connotated wrongly in too many heads (even for EV because what you really need to check is that it is issued to the right company).

Damn it, those people creating those safety belts. Now drug dealers use them too and they don't die when they run away with drugs in their fast cars !

Why is nobody complaining about the fraudulent domain names and the registries that enable them? This is where the problem starts. The certificate only proves that the owner has access to the domain, nothing more.

That's not going to do much, because subdomains are a thing. Aren't a lot of phishing domains like:


I think you misunderstand how domain validated certs work. DV certs have always been issued with the simple check that you own the domain (either by email or by DNS). The email check was always a bit scary because email is so readily forgeable.

damage they've done? Or damage phishers have done? Are we not to advance technology because some scumbags might abuse the technology?

0.02% is a lot less than I thought there'd be. There's also nothing stopping people from buying certificates from legacy CAs, do you have the stats on how many non-letsencrypt phishing certificates there are?

Do you really think the world is worse off because of Let's Encrypt?

What prevented Paypal phishing sites from buying certificates from other providers?

The part where it wasn't scalable because it wasn't free.

checks they're supposed to be performing. Some are ignoring them but being punished for it. http://www.bbc.com/news/technology-39365315

I'd definitely recommend clicking through that article and reading the source of the announcement:


> As captured in Chrome’s Root Certificate Policy, root certificate authorities are expected to perform a number of critical functions commensurate with the trust granted to them. This includes properly ensuring that domain control validation is performed for server certificates, to audit logs frequently for evidence of unauthorized issuance, and to protect their infrastructure in order to minimize the ability for the issuance of fraudulent certs.

> On the basis of the details publicly provided by Symantec, we do not believe that they have properly upheld these principles, and as such, have created significant risk for Google Chrome users. Symantec allowed at least four parties access to their infrastructure in a way to cause certificate issuance, did not sufficiently oversee these capabilities as required and expected, and when presented with evidence of these organizations’ failure to abide to the appropriate standard of care, failed to disclose such information in a timely manner or to identify the significance of the issues reported to them.

i.e. they apparently weren't even checking requesters controlled the domain they requested a certificate for in some instances!! And that's pretty much the only requirement for DV certificates?

My point is that the legacy ssl providers you're referring to were doing less checks than Lets Encrypt!

Both Mozilla[1] and Google[2] are on the record as saying that they don't believe CAs should be the ones to detect and block domains used for phishing and malware. Both are also platinum sponsors of Let's Encrypt.

[1]: https://groups.google.com/forum/#!msg/mozilla.dev.security.p...

[2]: https://groups.google.com/forum/#!msg/mozilla.dev.security.p...

DV certificates were never required to have different checks than what LE is doing, so what checks are you talking about?

Applications are open for YC Winter 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact