
I'm giving up on HPKP - el_duderino
https://scotthelme.co.uk/im-giving-up-on-hpkp/
======
skywhopper
The decision to use HPKP is about balancing risks. You're reducing the risk
that a compromised CA can issue a fake certificate for your site (the use of
which also requires either DNS compromise or some targeted MITM hijacking) in
exchange for increasing the risk that operational failure or the need to
rotate certificates more quickly than anticipated will block all users from
your site for a potentially long period of time.

The questions to ask are: how bad is the failure case, how likely is it, and
how hard is it to recover from? All three answers, IMO, weigh strongly in
favor of _not_ using HPKP.

An invalidly issued certificate is hard to make use of, except in targeted
situations (malicious wifi for DNS spoofing, or a compromise of trusted root
certs on the target machine), or for limited amounts of time (actual DNS
takeover). So the number of users affected would be small in either case. And
if there's a root CA out there granting invalid certificates, there's enough
interest in keeping the trusted roots clean that it would get patched up
pretty quickly anyway. So, the impact is low, the probability is low, and the
recovery is relatively quick.

But HPKP is rife with opportunities to screw yourself over unless you're
extremely careful. And a mistake can potentially affect all your users for
whatever period you configured the pinning to be for (which might be months or
longer if you don't think it through or just make a mistake). Recovering from
that is difficult or impossible.

So given those tradeoffs, the choice seems obvious to me. It's far easier for
attackers to just use a different domain name and count on people not
realizing or not even knowing which site they are visiting. That requires no
CA compromise, no trusted-root hacking, no DNS MITM, no wifi trolling. HPKP is
a problematic solution to a problem no one actually has.

~~~
schoen
> So the number of users affected would be small in either case. And if
> there's a root CA out there granting invalid certificates, there's enough
> interest in keeping the trusted roots clean that it would get patched up
> pretty quickly anyway.

However, HPKP is one of the mechanisms today that can best _detect_ this
activity happening. (In the future, CT with mandatory log inclusion proofs
should be better.) People don't just learn about misissuance events by magic,
but by having technologies that provide other channels for checking what
certificates exist in the wild or what certificates UAs should expect.

Also, some misissuance can be unintentional on the CA's part, so pinning can
conceivably also help clean up the CA's practices.

You might be right that these factors weight against HPKP most of the time,
but for the core misissuance-detection one, the situation is a bit more
complicated because of the way that HPKP itself helps with catching these
situations.

~~~
syrrim
There is a header one can set, as an alternative to HPKP, that will merely
notify you in the event of a misissuance, rather than block off your domain.
Perhaps that would be preferable?

~~~
schoen
Yes, the notification mode would probably often be sufficient in many cases
(and I think might be what caught the DigiNotar compromise?). I forgot what
happens if an attacker tries to block the notification from going through,
though.

------
arkadiyt
Lots of good HPKP discussion on infosec twitter as a result of this article.

I agree that HPKP in enforce mode is too dangerous to be valuable, but running
HPKP in report-only mode can be an effective canary against (poorly executed)
MITM attacks or other accidental server/router/ISP/etc misconfigurations.

------
Gasparila
This article (and others I have read disliking HPKP) generally take an all or
nothing approach. That is to say, they see HPKP as either "pin the key of my
certificate" or "don't use HPKP at all"

In reality, you can have a valid HPKP setup as long as any keys in your
certificate chain match the key in the HPKP header. This means that you can
pin the intermediate certificate (or even the root CA if you want) key. By
doing so, you are no longer vulnerable to losing your SSL keys, but now other
CAs cannot issue certificates for your site

~~~
viraptor
Or you get N certificates from different sources, pin them all, use 1, stick
the rest in (a) safe vault(s). Even if something really bad happens, you can
use one of the other ones until you cycle. Cost of N certificates will be <<
money you'd lose otherwise, and likely a rounding error in any company's
monthly spending. (if it's already live / trading)

~~~
pfg
HPKP is based on the public key, so you don't necessarily need to obtain
signed certificates in advance - generating a key pair and keeping the private
key safe would suffice.

There is some value in ensuring that a CA is willing to sign a certificate
using those keys in case something went wrong during the key generation (i.e.
a key size or curve that's not supported by the Web PKI), so it might be
considered a best practice to do that regardless.

~~~
viraptor
It's possible. But if you're in a disaster recovery situation, do you want to
add the step of granting the certificate as well? Do people on call have
access to the company credit card to get one?

It's easier to just get a full cert ahead of time.

------
SadWebDeveloper
Let's Encrypt set HPKP on suicide watch.

Let's Encrypt devs decided to put those short-lived certs (instead of the
long-term certs everyone was using) making it almost irrelevant to use that
"security header", almost every webdev knows the issues present on the current
cache-invalidation problems on every major browser.

~~~
vbezhenar
What do you mean? How HPKP and Let's Encrypt collide with each other? Let's
Encrypt is just another CA. You can use the same key for reissuing your
certificates, so HPKP header will stay valid.

~~~
SadWebDeveloper
[https://community.letsencrypt.org/t/hpkp-best-practices-
if-y...](https://community.letsencrypt.org/t/hpkp-best-practices-if-you-
choose-to-implement/4625)

[https://community.letsencrypt.org/t/official-hpkp-support-
fr...](https://community.letsencrypt.org/t/official-hpkp-support-from-lets-
encrypt/23753)

[https://github.com/certbot/certbot/issues/1611](https://github.com/certbot/certbot/issues/1611)

------
hdhzy
Scott, so take HPKP out of your securityheaders.io site. Encouraging people to
deploy this extremely sharp tool can bring only bad results.

~~~
GlitchMr
He did, on test.securityheaders.io. The change probably will move to actual
site (non-test version of it) soonish.

~~~
hdhzy
Excellent!

That also shows damage that can be done by people blindly perusing A+/100% on
security tests. The same goes to ssllabs that needs 4096 bit RSA keys (or
equivalent) to get 100%. Just... a bad idea.

------
KirinDave
Okay, but I still think mobile app apis served over SSL that don't pin to a
specific signer are making their lives way harder.

~~~
skywhopper
This is true. This is an actual case where pinning is a good idea, but this
sort of pinning can and should happen in the application code or config, by
restricting which CAs are trusted for a particular transaction. There's no
need for a generic HTTP header for this situation. Set your TLS negotiation
requirements in your app.

------
rrdharan
I'm very curious to hear what the community thinks of the dangers outlined
here.

I've been hesitant about setting up HPKP before for small unimportant personal
sites before since I have poor operational hygiene for them (I keep my
machines patched so I don't get pwned but I have lost SSL keys before due to
laziness or forgetfulness).

~~~
vbezhenar
It's hard for me to justify a technology which could do very serious damage to
a domain. I did the same, I set up HPKP for my domain and then I've lost my
keys (I forget to pay for hosting and server was deactivated). It's not a big
deal for me, I'm not really using this domain, but even with this throwaway
site I've had problems, when I uploaded some file and sent link to my friend,
but he wasn't able to download it, because he was there before.

Blackmailing domain owners with HPKP is a real threat as well. It'll happen,
sooner or later, servers are getting owned all the time. Defacing front page
is visible. Setting additional header is not.

~~~
Ajedi32
> I've lost my keys (I forget to pay for hosting and server was deactivated)

Both of them? You mean you were storing your backup key on the same server the
main key was on? Doesn't that kinda defeat the purpose of having a backup key
in the first place?

~~~
skywhopper
Mistakes happen. New certs and keys can be easily acquired if you lose the
key. But HPKP prevents you from fixing this simple mistake in order to guard
against a problem that's extremely unlikely.

------
kardos
Re: HPKP suicide, why not pin a handful of 'recovery' keys that you can use
when things go sour? Stored off-server or offline so in the hack scenario you
don't lose them.

~~~
astrodust
In a "hack scenario" theoretically everything can be compromised.

~~~
Ajedi32
Sure, in theory. Realistically though, I think any scenario that involves
compromising both the production server _and_ a USB drive locked in a safe is
pretty unlikely.

~~~
pfg
The interesting bit about RansomPKP is that it works independent of whether
your HPKP pinning strategy is sound. In fact, it works even if you don't use
HPKP at all. All the attacker needs is the ability to modify your HTTP headers
and to either have access to your existing private key, or the ability to
change the private key to one under their control. From that point on, it's
all about how long it'll take you to notice the modified header. If it's live
long enough for a sufficiently large percentage of your visitors to hit the
site, the damage is done.

One thing to take away from this is that RansomPKP is not so much an argument
you can use when deciding whether you should deploy HPKP for a particular
site, but rather something that's relevant for deciding whether browsers
should continue to support HPKP in its current form at all. I could see how a
couple of large RansomPKP cases could very well cause a change on that front.

~~~
buu700
Actually, more than just being independent of your pinning strategy, one of
the partial mitigations that Bryant (@eganist) and I recommended when
introducing RansomPKP was to _enable_ HPKP, since it would both block existing
users of the target site from pinning the new malicious headers and cause
alarm bells to be set off early.

Like you said, RansomPKP is more impactful to the ecosystem-wide decision of
whether HPKP should exist than to whether an individual should use it (given
the precondition that it exists). That said, malicious pinning was explicitly
discussed and noted as a concern during the standardization process, so
RansomPKP doesn't add much to that ecosystem-wide decision; it's just a fun
application of HPKP Suicide that takes the core malicious pinning idea to a
logical conclusion, and isn't even necessarily the worst way to abuse a
compromised web server. There are also much more positive applications of HPKP
Suicide, e.g. my startup Cyph's in-browser code signing layer.

Anyway, I wouldn't use any of this to argue against the very existence of
HPKP, but I agree with Scott removing it from a tool that at least partially
targets non-experts.

~~~
Ajedi32
> one of the partial mitigations that Bryant (@eganist) and I recommended when
> introducing RansomPKP was to enable HPKP, since it would both block existing
> users of the target site from pinning the new malicious headers[...]

Huh? How? If the header changes, don't browsers automatically update their
pins next time the user visits that page?

> [...]and cause alarm bells to be set off early

I don't get this either. If an attacker enabling HPKP wouldn't set off any
alarm bells, why would changing an existing HPKP header do that?

~~~
buu700
At the time, if I recall correctly, we'd tested both Chrome and Firefox and
confirmed that any HPKP header that didn't match the key currently in use
would be ignored. Because of this, the only way we could see to replace an
existing pinned key would have been to remove or replace the existing header
and keep that change intact for the duration of the header's max age while
remaining undetected the whole time (which would be unlikely when faced with a
competent InfoSec team, as you implied).

As far as early alarm bells, sorry, that was a bit vague. I was referring to
not just ambiguous warning signs that would be caught by careful monitoring,
but the inevitable stream of reports from end users about the big scary TLS
pinning error screen in the browser, should the attacker decide to immediately
push out the malicious keys/headers without waiting out the old ones' expiry.

~~~
Ajedi32
Ah okay, I see what you're saying. If the HPKP header is set, the attacker
can't transition the server over to using a new key the attacker controls
until _after_ the max-age expires, otherwise they'd risk tipping their hand
when users who still have the old set of keys pinned start getting key pinning
errors and complaining to the site owner. (And they can't remove the original
key from the set of pinned keys until after they make that transition, since
key pinning headers that don't include the key you're currently using are
ignored.)

~~~
buu700
Yep, that's exactly it. There are a bunch of possible
permutations/optimizations, like initially keeping the original header with
the max age changed to 1 for a shorter period of time (until you're confident
that the number of user reports will be small enough not to get on anyone
important's radar before it's too late, and/or maybe you have some more
complicated logic to track which clients do and don't have the altered
header); but in any case the general point we made in the talk was just that
enabling HPKP (among a few other things) was a net positive if you were
worried about getting hit by RansomPKP today, not that it would entirely
prevent the attack.

------
perpetualcrayon
Does anyone here know how (or if) each of the major browsers caps max age? It
appears the specification provides a hint that it's up to UA whether to cap
max age.

If the browsers aren't capping max age, could this affect future owners of a
domain had the HPKP max age been set to several years after their ownership of
the domain expired?

~~~
hdhzy
They do cap: 10 weeks in Chrome, 60 days in Firefox [0] but you can still
poison future owner's domain.

[0]:
[https://mobile.twitter.com/spazef0rze/status/900698403629391...](https://mobile.twitter.com/spazef0rze/status/900698403629391877)

------
mozumder
How is HPKP supposed to work when you change certificates? The new
certificates don't have HPKP and the site still relies on old HPKP.

Letsencrypt certificates only lasts 90 days. It's a real pain-in-the-ass to
figure out a process that allows HPKP with certificates that expire every 90
days.

~~~
ivanr
If you continue to use the same public key, your existing pin will work with
your new certificate. So there's nothing to do. IIRC, many (all?) LE clients
generate new keys by default, so this is something you need to plan for in
advance.

If you want to change your public key, you have to obtain your certificate in
advance, then introduce its pin into your HPKP configuration at least N days
before the certificate switch. In this case, N is your maximum HPKP policy
duration. With this process you ensure that, when you do switch the
certificates, all your users will have the correct pin for it. Whatever HPKP
policy you had N+1 days prior, it will have expired by then. You probably want
some additional safety margin there too.

------
hyperfekt
Am I mistaken to think that the HKPKP attacks could be mitigated to a certain
degree by implementing delayed pinning - meaning a pin must be announced for a
certain amount of time before it goes into effect, allowing the owner to
notice and counteract a malicious pinning.

~~~
georgyo
HKPKP is a header in http request, there is no where to announce it early.

~~~
icebraining
The idea is that browsers would see that header, but only enforce the pin if
the header was kept for a certain period.

