> Let's Encrypt finally solved that, but at the same time effectively made SSL certs mostly worthless for identity purposes
Which was and is reasonable, given that encryption mitigates risks that can't be mitigated otherwise, and is therefore a more pressing concern for security -- you can take many other measures to verify who the party on the other end of a connection is, but once unencrypted traffic exits your network, there's not much you can do to prevent snooping.
It again goes back to muddling up these two separate concerns. If we used CA-issued certificates only for identity validation, we could have optimized them for that purpose and not diminished their value because the need for ubiquitous encryption was a higher priority. Meanwhile, we could have just had web servers auto-generating encryption keys (a la SSH, as you point out), without being shackled to the trust hierarchy, and had universal adoption of HTTPS 20 years ago. Instead, we got a single solution to two different problems that is still forcing us to compromise one for the other.
Which was and is reasonable, given that encryption mitigates risks that can't be mitigated otherwise, and is therefore a more pressing concern for security -- you can take many other measures to verify who the party on the other end of a connection is, but once unencrypted traffic exits your network, there's not much you can do to prevent snooping.
It again goes back to muddling up these two separate concerns. If we used CA-issued certificates only for identity validation, we could have optimized them for that purpose and not diminished their value because the need for ubiquitous encryption was a higher priority. Meanwhile, we could have just had web servers auto-generating encryption keys (a la SSH, as you point out), without being shackled to the trust hierarchy, and had universal adoption of HTTPS 20 years ago. Instead, we got a single solution to two different problems that is still forcing us to compromise one for the other.