
Let's Encrypt has turned on stricter validation requirements - mmoez
https://community.letsencrypt.org/t/acme-v1-v2-validating-challenges-from-multiple-network-vantage-points/112253
======
jaas
The blog post is probably more understandable and relevant for readers here:

[https://letsencrypt.org/2020/02/19/multi-perspective-
validat...](https://letsencrypt.org/2020/02/19/multi-perspective-
validation.html)

Maybe we can get the link changed?

~~~
jsjddbbwj
I don't see how that less technical, more full of fluff post is better for
readers of this website

~~~
wpietri
I'm fine with keeping the original link. But the original post doesn't
actually explain why this is happening, while the suggested replacement
explains the problem.

------
hedora
Does anyone know of a “set it and forget it” alternative to Let’s Encrypt?

I’m all for making things more secure, but they’ve broken all of my certs in
the last 12 months (in different ways, at different times), and I’m sick of
it.

~~~
sandGorgon
a regular certificate.

For example you could buy a Sectigo (previously Comodo) certificate for 2
years for like 17$ or so. Wildcard for 2 years is like 100$.

Its worth less than a broken certificate.

~~~
devrand
What do you use to automatically rotate the certificates from Sectigo? This
doesn't seem to be a set it and forget it solution.

Also, with Sectigo you're more likely to get an actual broken certificate.
They misissued nearly every certificate starting from 2002 until late 2019
[1].

[1]
[https://bugzilla.mozilla.org/show_bug.cgi?id=1593776](https://bugzilla.mozilla.org/show_bug.cgi?id=1593776)

~~~
sandGorgon
We rotate them _manually_ once every two years. It works. No need for a
certbot. Simpler infrastructure

Sure about the broken, but the alternatives are worse with possible leaked
private keys.

~~~
devrand
How is that simpler? That seems like it doesn't scale and incredibly error
prone.

If you only have one certificate you may get away with it. But if once you
have hundreds or thousands this is absolutely going to break down the human
factor.

And even if you do have a single (or few) certificate(s), there are other
factors that are going to complicate maintaining this system:

    
    
      * What if a certificate needs to be revoked by your CA? Generally CAs are obligated to revoke certificates within tight deadlines (ex. 24hr for key compromise). That doesn't give a human a lot of time to replace the certificate.
      * What's going to happen when 2-year certs are no long available? Ballot SC-22 failed, but it would've reduced certificate lifetimes to 1 year. Some CAs are moving in this direction anyway, and it's worth noting that Sectigo supported this ballot.
      * What happens when the person responsible for renewing them leaves the company and forgets to hand-off the responsibility?
    

I almost see the infrequency of certificate rotation as a negative since it
means the process is infrequently tested and easy to forget about.

Sure tools like cerbot can break, but if you know that it's renewing
certificates 30 days out then setup alerting for whenever a certificate
expires in less than 30 days. You should have this alerting anyway in case the
human responsible for manual rotation forgets.

If you ended up in a state where you were serving an expired certificate then
the key issue is your alerting.

~~~
devrand
And what timing! Just today it was announced certs trusted by Safari issued
after Sep. 1st must have a lifetime of 1 year [1]

[1]
[https://twitter.com/chosensecurity/status/123025334823601357...](https://twitter.com/chosensecurity/status/1230253348236013570?s=19)

------
londons_explore
This is a good start...

But I think it would be far better for them to focus on _alerting_ webmasters
if someone does manage to get a new certificate issued for a domain before the
old one expires.

Certbot should reference the old certificate when doing a renewal. If someone
registers a new certificate while an old one is valid and without referencing
the old one, the owner of the old certificate should be notified loudly (sms,
different-domain email, etc). Same if they register a certificate through a
different provider.

Today all of the above is possible with certificate transparency logs, but
nobody looks in them, so they're useless.

~~~
waste_monk
>Today all of the above is possible with certificate transparency logs, but
nobody looks in them, so they're useless.

I check mine once or twice a month manually, but it is pretty trivial to
monitor it automatically as well, e.g. there are APIs for crt.sh or they even
offer direct public read access to their database.

I believe certificate users should remain responsble for their own monitoring.
Alerting as you say would be very annoying since you couldn't preemptively
replace certs without getting alerted unecesarily, and it would divert Let's
Encrypt developer resources away from more useful projects.

~~~
technion
I built this project to make such automated monitoring easier:

[https://ctadvisor.lolware.net](https://ctadvisor.lolware.net)

~~~
smartbit
Love the site, great sense of humor.

Wish you all the best with this project.

~~~
technion
Many thanks! And thank you to the people that signed up to try it out.

------
mittalprat
For readers interested in understanding the technical details about potential
BGP attacks on domain validation, which serve as a motivation for Let's
Encrypt's multi-perspective validation deployment, see the following paper
from USENIX Security:
[https://www.usenix.org/conference/usenixsecurity18/presentat...](https://www.usenix.org/conference/usenixsecurity18/presentation/birge-
lee)

~~~
dgacmu
And for background on the multiple perspectives validation approach:

[http://www.cs.cmu.edu/~dga/papers/perspectives-
usenix2008.pd...](http://www.cs.cmu.edu/~dga/papers/perspectives-
usenix2008.pdf)

~~~
movedx
This was excellent. Thanks for sharing.

Think about the contents of that video/PDF for a second. The guys/gals doing
that bit of research at the university are clearly smart, but imagine what a
state level actor can do?

------
spydum
I don't see how this resolves BGP hijacking attacks? If I announce a more
specific route all four locations should still land on my hijacked network...
Or is this trying to race the BGP propagation?

~~~
jaas
It doesn't "resolve" BGP attacks. The point is an attacker would have to pull
off three or four successful attacks at once, which is harder than pulling off
just one, especially if they hope to go unnoticed.

~~~
tialaramex
Or of course for smaller targets their attack is just much closer to the
target than to any of the multiple viewpoints, and so this mitigation makes no
difference to them whatsoever.

We shall see how well this works in practice. Assuming it's relatively cheap
it's harmless to at least try.

------
Rebles
I may be naive, but it seems like it might be more secure if the first step
was to deploy a self-signed cert on the server, step 2, give Let's Encrypt the
public key of the self signed cert, so Let's Encrypt can validate who you are,
then proceed with Let's Encrypt's regular validation process, obviously
replacing your self-signed cert with the one issued by Let's Encrypt at the
end.

~~~
smw
Wouldn't an attacker be able to create a self-singed cert just as easily?

~~~
tialaramex
Self-signed. Yes, an attacker would not ordinarily find this harder to pass
than the http-01 challenge today. Validation using this approach was method
3.2.2.4.9 ("Test Certificate") and is no longer permitted for new issuance
under current Baseline Requirements.

Let's Encrypt offers three ACME methods which implement 3.2.2.4.6 ("Agreed
Upon Change to Website"), 3.2.2.4.7 ("DNS Change") and 3.2.2.4.10 ("TLS Using
a Random Number").

~~~
movedx
> 3.2.2.4.9 > Baseline Requirements

Where can I find these details? Sorry if I'm being a bit dense here.

~~~
tialaramex
The CA/Browser Forum publishes the Baseline Requirements to their web site

[https://cabforum.org/baseline-requirements-
documents/](https://cabforum.org/baseline-requirements-documents/)

In recent years the BRs are using RFC 3647 structure. This RFC gives an
outline for how to write policy documents for PKIX (X.509 Public Key
Infrastructure for the Internet) and rather than wrestle with each
organisation having its own preferred way to organise much the same
information the trend is to require RFC 3647, so you know the stuff about
names will be in section 3 for example

The RFC 3647 structure doesn't break down as far as 3.2.2.4 but 3.2.2 is where
people explain how they're going to validate organisation names, and so in the
Baseline Requirements 3.2.2.4 is where the "Ten Blessed Methods" are
described, the authorised means by which public CAs can determine if the name
you want a certificate for is really yours.

~~~
movedx
Thanks for sharing that, friend. Appreciated.

------
sysashi
I've also received a notification email about my outdated acme client, thanks!

------
oblib
We'll see how it works in practice. After reading the intro to what they're
doing I think I'll update a few certs before they expire to make sure it
works, but from what I've read I don't anticipate any problems.

LE has been great for me.

------
jve
FYI, Last week, before I unblocked my firewall, when I tried to get certs from
LE, it tried to knock me some 5 or 6 times from different IPs. Checked some
ASNs, one was from AMAZON, the other one I don't remember.

------
creeble
TL;dr - can this be fixed by updating to the latest certbot package?

~~~
progval
You don't need any update.

------
eric_b
It's already painful to get Let's Encrypt set up in a web farm scenario. This
won't make it easier.

~~~
snapetom
If you or your company has enough money for a web farm, you should just buy
your own cert.

~~~
GreenJelloShot
What? Web farms are automatically expensive? A web farm can literally be two
Raspberry Pis behind a simple load balancer. What if this isn't "my company"?
What if I run a web farm at home for fun?

Besides, your argument could be used to justify any price hike! "If you can
afford X, then you can afford Y!"

~~~
nine_k
Why isn't your LB terminate TLS?

------
polyphonicist
Does this require the user to make any changes?

I run a simple static website served with Nginx. I would like to know if this
change has any impact on me.

~~~
mcpherrinm
No changes are required, unless you had some kind of IP restriction on who
your server may be contacted by. Which your simple static site is probably
fine with.

------
Avamander
The first paragraph already highlights the issue I have with it. I don't want
to loosen my firewall, some countries and their IP ranges just need not access
my server.

~~~
dspillett
I have my certs issued to a VM this is not accessible via HTTP(S) anyway, just
DNS. It runs a small DNS server which answers to _acme-challenge.* requests
"forwarded" via CNAME. As that is the only part of is visible to the rest it
has a very small security surface to worry about. It then pushes out the keys
& certs to the places that need them (directly in the case of local machines,
less directly for some others).

I set that up originally for a wildcard, as http validation is not supported
for them and I didn't fancy mucking about automating updating the main bind
instances, but it is convenient for the others too and will be unaffected by
these changes.

Not a perfect solution for everyone of course, copies of all the keys are in
one place for one thing, and it all in one place could be bad or good for
maintenance (single point of failure, but single service to monitor &
maintain), but worth considering if you expect problems with http validation.

 _> some countries and their IP ranges just need not access my server_

I'm guessing that they won't be using locations that are commonly blocked in
this way anyway. And if they do happen to use one, you may be fine as only two
of the three external checks need to be responded to.

~~~
nomercy400
I've read about this a few times now, but have been unable to find a good
resource on how to set such a VM up. Do you have a link or resource somewhere
so that I can at least get started?

Still unexperienced with letsencrypt, but I know enough that I cannot use the
standard way.

~~~
throw0101a
> _I 've read about this a few times now, but have been unable to find a good
> resource on how to set such a VM up._

You first set up a VM and set up your favourite authoritative DNS software on
it: popular choices are ISC's BIND and NLnet's NSD. Either will do. Call it
(e.g.) _ns-dnsauth.mydomain.com_ , which is Internet accessible only on udp/53
and tcp/53.

You have to then configure that DNS server to serve the domain (e.g.)
_dnsauth.mydomain.com_.

Next you configure the DNS server software to allow dynamic updates. For ISC
BIND, you can set up (crypto) keys and use the _nsupdate(1)_ utility:

* [https://www.zytrax.com/books/dns/ch7/xfer.html#allow-update](https://www.zytrax.com/books/dns/ch7/xfer.html#allow-update)

* [https://dan.langille.org/2017/05/31/creating-a-txt-only-nsup...](https://dan.langille.org/2017/05/31/creating-a-txt-only-nsupdate-connection-for-lets-encrypt/)

Point your public/external DNS records to your delegated-auth server by having
(say) __acme-challenge.www.mydomain.com_ be a CNAME to (say) __acme-
challenge.www.dnsauth..._. LE will follow the CNAME and try to do the
verification against the record in _dnsauth_ sub-domain that lives on the _ns-
dnsauth_ VM.

Then you have your LE/ACME client(s) run a hook script to publish (and
cleanup) the _dns-01_ TXT challenge records:

* [https://dan.langille.org/2017/07/04/acme-sh-getting-free-ssl...](https://dan.langille.org/2017/07/04/acme-sh-getting-free-ssl-certificates-installation-configuration-on-freebsd/)

* [https://github.com/dehydrated-io/dehydrated/wiki/example-dns...](https://github.com/dehydrated-io/dehydrated/wiki/example-dns-01-nsupdate-script)

The LE client goes to the LE API, gets a verification token/nonce, executes
the the hook script to push the TXT record to _ns-dnsauth_ , the LE folks
verify the record, the LE client (ideally) cleans up the TXT record, receives
the cert for the LE API, puts it in the correct path and restarts your (web)
service(s).

Someone actually wrote a limited-functionality DNS server that allows for
pushing of records via a REST API for this purpose:

* [https://github.com/joohoi/acme-dns](https://github.com/joohoi/acme-dns)

This way the 'heavier' BIND/NSD software doesn't have to be used, as those
have more features than are needed.

~~~
anonsivalley652
For publicly-accessible infrastructure:

4-6 instances of pdns authoritative for a domain, and pdns recursor running
locally for each box. And Cloudflare free tier while revenue can't justify
rolling-out Varnish and other locally-deployed capacity/DDoS mitigations.

It may also be a better idea to push DNS updates via configuration management
or driven from something like Envoy so there's a history and a single-source-
of-truth (SSOT) to point-to rather than multiple people doing manual
tinkering, which is a labor-intensive, antiquated approach.

~~~
throw0101a
Is "pdns" PowerDNS?

* [https://doc.powerdns.com/authoritative/dnsupdate.html](https://doc.powerdns.com/authoritative/dnsupdate.html)

If we're just talking about issuing certs, I don't know why one need 4-6
instances for serving the _dnsauth_ sub-domain.

------
MrStonedOne
Too bad I can't read them because the site doesn't load at all with 3rd party
scripts disabled.

~~~
thenewnewguy
Works fine for me, uMatrix with javascript disabled by default.

Edit: Ah, upon testing it breaks if you have 1st party JS allowed but not 3rd
party. This is pretty reasonable in my opinion.

~~~
azdle
For me it seems to also require that you turn on "Spoof <noscript> tags" in
uMatrix.

