The self signed cert will need to be rotated from time to time, without being signed by previous cert because e.g. private key was lost or because private key is known to have been compromised. How will your browser, connected to cafe wifi, know that change in self signed cert since last connection is legitimate vs MiTM by wifi AP in cafe?
The CA provides multi perspective check from servers connected to internet backbone, where network hijack is harder than at cafe wifi AP.
The CA system also provides dozens of organizations all over the world to issue certificates for any domain and to hand them out to pretty much anyone.
If Microsoft included a self-signed cert in my Windows install for itself, I would know my communications with Microsoft came from whoever made my Windows operating system. Meanwhile, with PKI, dozens of companies can technically generate certificates for microsoft.com and we're just all hoping none of them do it without getting caught.
You haven't answered the question. There are flaws with the CA system, yes. But replacing it with self-signed certificates creates new problems. How do you address them?
There are mitigations in various stages of progress for this, like certificate transparency, CAA records, and the removal of CAs who violate best practices through either malice or stupidity.
Those changes are being largely driven by Google/Mozilla/etc, via enforcement around what CAs must do in order to be part of the root of trust.
Switching to self-signed certs doesn’t remove any problems. With current PKI, dozens of companies can generate certificates for my website which will be trusted by user browsers. Without PKI, literally anyone can generate a self-signed cert for my website, and there’s no concept of which certs are valid, unless somehow everybody finds a way to share which certs are theirs (and solving that is generally called “PKI”).
EV doesn’t allow self-signed certs to work either, or viably replace DV certs for any threat models, because it’s just as easy to register a similar-sounding company name as to register a similar-looking domain name. Arguably it’s easier, because you can actually register exactly the same name, just in a different jurisdiction.
Your OS or an app talking to their mothership can and do use cert pinning. What about all the millions of websites?
A rogue/compromised CA will be caught by certificate transparency and distrusted.
Also, as 'tptacek noted, you haven't explained how to securely use self signed certs to protect against attacker at the WiFi AP, which is a very accessible kind of attack (compared to compromising a CA, a registrar, a datacenter or internet backbone traffic).
The CA provides multi perspective check from servers connected to internet backbone, where network hijack is harder than at cafe wifi AP.