Incidentally, this is the same problem as public key distribution. You need a trusted channel to receive public keys, and a trusted channel to know whether to use a public key. Why can't these be the same channel? Right now we have HSTS preloading for the latter, but in that case why not preload certificates (or hashes thereof) too?
Then we can finally cut out the middle-men and realize the truth: that the browser is the ultimate certificate authority.
That is not worst case scenario. If someone can force http, they can also inject malicious code into the stream and do anything from bank transfers to create botnets. With the worst case scenario of always https being DoS, and worst case scenario of allowing http is code injection, I would prefer deprecating http in favor of https.
• HTTP is redundant and costly when you're already in some other tunnel: a pre-negotiated IPSec tunnel for port 80 traffic to a given peer (e.g. a load balancer to its backend); talking directly to an HTTP proxy sitting on the jump box you're VPNed or SSH tunnelled into; etc.
• HTTP is actually a great wire protocol for non-networked RPC, such as between Nginx and your application server, running on the same box, over a Unix socket. FCGI, WSGI, etc. are just half-assed implementations of HTTP; you may as well just use HTTP. (Though the non-front-of-line-blocking benefits of HTTP2 RPC would be even better here, for green-threaded runtimes that can C10K.)
I do agree, though, that unencrypted HTTP can likely be deprecated for web browsers. The browser-addressible web is really a pretty strictly-bounded subset of the web as a whole, and we should strive to make it safe to browse.
That being said, such statements put me in mind of a future where your browser literally is not allowed to talk to all those old servers from 1997 that are still hosting whatever they were hosting back then. Instead, all requests for those "legacy" domains that nobody's updating any more would have to go to some trusted mirroring site served over HTTPS, like the Internet Archive. (The spidering logic for such "legacy mirroring" would also have to be slightly different from today's "latest mirroring" logic: if the IA's spider got MITMed to see something else, it should "doubt" the new version based on how long the previous site endured without change, and if its confidence is low enough, just continue showing people the old version.)
Is that a future we want? I'm honestly not sure.
What matters is that the web of trust associated with that key is sound (that is, you have assurance that the key belongs to whom you think it does), and that the integrity of the private key has been maintained.
The first of those problems is difficult, but not intractable. The second problem is rather difficult, especially in the case of persistent data, though the core requirement is that the key was valid when a message was generated, if you're looking at the sender of information. For your own information, you are relying on the recipient to maintain integrity over their private (decryption) key going forward, such that the data you'd transmitted remains encrypted against all others.
The first problem you point out, that any encrypted channel is not necessarily a secure channel, is valid, though given your misunderstanding on subsequent points I'm not sure how well that applies to this discussion (I still need to RTFA).
btrask wasn't saying that encryption is necessary for key distribution; he/she was saying that HTTPS guarantees identity and integrity, both of which are necessary to trust a key.
> What matters is that the web of trust associated with that key is sound (that is, you have assurance that the key belongs to whom you think it does), and that the integrity of the private key has been maintained.
That's a possible alternative to btrask's proposal, though you're equating "assurance that the key belongs to whom you think it does" with "web of trust". btrask's proposal is a special case of that, in which the web of trust is simply the sender.
> The first problem you point out, that any encrypted channel is not necessarily a secure channel
Correct, but not what btrask said. The first problem he pointed out was the fact that clients need to know whether a host expects secure communication before ever connecting to it.
> though given your misunderstanding on subsequent points I'm not sure how well that applies to this discussion
That's not very nice.
I said that an encrypted channel could be used, and that it might not be, but that if used encryption would largely serve as a protection to the requestor, who might otherwise be subject to traffic and/or interest analysis based on the specific keys they requested, which could be presumed to be of interest, or signing keys (I'm thinking PGP protocol here) of keys of interest. Either piece of information would reduce search space for an Eve.
I'm not equating trust of keys to web of trust, I'm stating that in existing (PKI/PGP) protocols, that is the assurance mechanism. And it is independent of either trust OR encryption of the key delivery channel itself.
There seems to be a rather profound difficulty in distinguishing what I've said with what I've said btrask said. I'm not sure how I could be clearer, but I'm open to pointers.
Maybe I misunderstand (though I also think I understand teh issues involved pretty well), or maybe one or the other or both of us are communicating poorly.
How would you distinguish a trusted, encrypted, and untrusted channels, say?
Relevant to my original post, information about whether the connection should be encrypted also merely needs to be authenticated, not encrypted itself. Of course, the HSTS preloading site uses HTTPS (with encryption) because it's easy and why not.
I'm reading the auth and channel as independent. Auth is something of a metachannel, perhaps.
Yes, that means verifying your keys, and probably through an out-of-band method.
Can you explain what you mean by this? Genuinely curious to know how it can lead to DoS?
In general, caching is a big problem with the naive approach to "HTTPS everywhere." A mechanism to deliver signed cacheable payloads would be great, so that static assets etc. can continue to be edge-cached.
The most annoying thing about HTTPS everywhere is that it ruins cacheability. This is a problem the distros solved ages ago by signing their content but acknowledging it's mostly pointless hiding it in transit.
But its absurd that in HTTP2 we have out of the box encryption, but we don't have a mechanism for doing authenticated caching.
We don't have a trusted internet. Not when a country on the other side of the world can mis-configure BGP and re-route all traffic through them. Not when our ISPs intercept and modify our traffic. Not when there are nearly 10x as many "trusted" root certificates as there are nations in our world.
The internet is the wild west, and we need to protect our computers from it. Currently, encryption is our best bet for doing that. If edge caching is a casualty, then so be it.
If someone can come up with a method for protecting content from end-to-end while keeping it secure against tampering and eavesdropping (because this too matters, both to us in the first world and the majority of others who are not), then let's start getting it put in place.
Your statement can be misinterpreted to imply that merely by encrypting all traffic, such analysis can be prevented. There's plenty of metadata in a typical encrypted page load that can be used to do so.
For example, the view-discussion page might download three static assets, a js file, and two CSS files, one small and one large, whereas the post-comment page might load zero assets and js files, and one small CSS file.
Point being, making a privacy-protecting website takes careful planning even when fully encrypted. As such, it'd be great to have tools (such as signed content) available for performance optimization. Sure, naive usage might lead to attack vectors, but naive usage of HTTPS already leafs to many such attack vectors anyways.
Why not expect it to be secure? Connect to https before http.
It would be trivial for any man-in-the-middle to block https and server http.
The main problem is preventing downgrade attacks. With mail it is easy to just remember the setting for every server. Not so with websites.
IMHO, the feature would need to be implemented as some have suggested, by enabling any website to transmit securely or insecurely, but for the web browser to request a secure TLS connection first (trying HTTP and HTTPS to reduce incompatibilities) and if a website appeared to have issues, then try insecure connections. If an insecure page were to be served, the browser should indicate this with a broken padlock.
Furthermore, I believe that browsers should warn when any data is input, e.g. clicking items that cause JS calls or text is typed - this strict implementation is important. Single page JS applications have made it possible to send any input data via JSON, we cannot only warn the user on a form submission, since it would be very possible to capture details via AJAX. E.g. If I were impersonating an e-commerce solution, I could hope the user would not notice the padlock and use AJAX to send the data preventing any form submission warnings. This would be annoying for users when they were using such websites regularly, but this would be a good thing - pressuring websites that handle user inputs to act responsibly and use encryption via TLS.
Then it would be good to remember this setting and always pull the HTTPS.
Tim Berners-Lee is certainly an authority in the area, but I (an amateur) fail to see any major problem here, let alone one that "completely breaks the web".
Can someone illustrate a use case where either this fatal link-breaking cannot be solved by a simple HTTP->HTTPS redirect, or any other scenario where the user is so much worse off?
In a way it is arguably a greater threat to the integrity for the web than anything else in its history. The underlying speeds of connection of increased from 300bps to 300Gbps, IPv4 has being moved to IpV6, but none of this breaks the web of links in so doing.
I'd venture to say that IPv6 probably wishes it had the traction that HTTPS Everywhere has...
To be more specific, I'm referring to the "Don't break the Web" section in the article.
This works fine if you use anchors without protocols in your html:
It's a simple fix to whitelist the one website though. It doesn't break the Internet. It breaks sometimes, for some users, and is trivially fixable when it does break.
"Fundamentally breaking the internet", to me, is something that actually breaks the usability of the internet in a non-trivial-to-fix way for the end user where the end user isn't even in control of the fix. That's breaking the web.
Failing to support IE5 is "breaking the web" in the same way Https Everywhere breaks the web. In a way that is to be fixed on the user-end.
(Although sites that fail to serve over https:// should fix their site)
HTTP and HTTPS use different ports (TCP 80 and 443). You can run one web server application on the HTTP port serving content A, and a completely different web server application on the HTTPS port serving content B. With firewalls doing NAT port translation this could even result in HTTP requests going to a completely different machine than the HTTPS requests.
From a non-tech reason, there are some types of sites that the HTTPS content should never be available on a HTTP site. For example an online payment form. In such cases a sensible website will either disable HTTP entirely (and use a subdomain for secure content, rather than the top leve/ www. domain), or have a basic HTTP site that transparently redirects to the HTTPS version.
It only takes 1 server to serve differing content through each protocol to break things for the user
(I do agree that it's contrary to expectations, but I'm sure this doesn't stop sites from relying on it.)
The questions of secure document transfer and/or interchanges are:
1. Am I talking to the party I intended to?
2. Is the communication free from third-party interception?
3. Is the message itself originated by the party I intended?
4. Are the contents of that message as originally intended by the author?
(Possibly more, but those strike me as the Big Four.)
There are various ways for this to fail, and there are different and independent assurances which can be afforded. I remeber the first time I heard phrases to the effect of "you can trust our secure webserver" in the context of commercial transactions, and cringed.
The present HTTP / HTTPS split addresses only a subset of these concerns, and few of them well, whilst breaking multiple elements of functionality.
I will note that TBL seems to be concerned over the expiration of old, previously-valid URLs. To that I can only say that this appears to be a lost battle. The duration of a contemporary URL is on the order of 40-45 days, I think from the Internet Archive. That's scarcely longer than an old-school Usenet post might be relied on to persist online, and suggests to me that perhaps the successor to Usenet is the Web, with origins and various archival services (archive.org, archive.is, the NSA, ...) providing robust storage needs to various audiences.
I can see an argument for having DANE-like records include an HSTS instruction, but nothing like that is mentioned in the article.
There are tons of non-HTTPS websites out there. A myriad of forsaken ones that are still running because no one had remembered to do anything to them, and a myriad of ones whose sysadmins just don't care about TLS at all.
A non-obtrusive "insecure connection" warning is probably going to happen quite soon, but I just don't see any chance of mass HTTPS migration besides the high-profile sites, newborn sites and geeks that would stand for the cause.
Otherwise, a pretty large fraction of the WWW is going to be lost.
I'm also a big fan of Let's encrypt, democratizing SSL certificates is only a good thing.
"There follow some thoughts following many recent discussions of "HTTPS Everywhere" and points west."
What does "and points west" mean here?
I think it's based on the expression "all points west." Can't find a good source at the moment. "Points" is used as a plural noun there, not a verb. So, I think the expression "all points west" means "all locations to the west," and used metaphorically here to mean "all things beyond."
So I guess it's a metaphor for the indefinite beyond that makes sense to people who live on the east coast of the US and ever take the train.
I have no idea what "and points west" means in this context.
Browsers should connect to port 80 and perform a GET for /tls-cert with an Accept: header listing all the certificate formats it knows and the applicable Host: header.
The server would respond with the certificate for that host-name and the browser would validate it. After that, the HTTP connection would switch into TLS mode using the key in that certificate.
If the server responds to this initial request with 404 or some other error, the browser either shows an error or continues in insecure mode.
This would have the benefit of bringing that initial certificate negotiation outside of the TLS black-box. For years, TLS deployment was held back because you couldn't have multiple domains on a single IP, long after the problem had been solved for HTTP.
Later, SNI was added to TLS, but the change wouldn't be rolled out to Windows XP users (except Firefox which used its own TLS implementation).
By using HTTP, you'd have the Host: header right away and could even introduce new certificate formats by looking at the Accept: header. This sort of thing is built into HTTP but had to be retrofitted with much pain and anguish into TLS.
[b] Is the only way to obtain the full preload list to extract it out of Chromium or Firefox source code? 
If it were a HSTS preload registry service and API, that'd be different, but it's not.
With this approach we can have it for big sites, or important sites but we want it everywhere.
There's no sense in which you "cannot" preload every site with an SSL certificate. You absolutely can, and it would work totally fine. We can talk about whether there are better designs, but preloading everything is definitely a realistic option.
It's also an option that works today. If we figure out a better solution in the future (DNSSEC? Bloom filters and OCSP responders?), we can seamlessly transition the current preload list to it, but we're also getting the security advantage of the preload list immediately.