Putting aside the whole issue of delegating trust via certificates, the simpler answer is that encryption is hard, and thereby expensive. For many purposes, both service provider and client continue to consider it an unnecessary expense.
Only recently, with the spread of wireless networks, has the threat of en masse MITM and eavesdropping attacks become realistic. For a long time, it took serious knowledge to pull these things off on a wired LAN or within the walls of an ISP.
You could say the same about phone calls--why isn't the POTS network encrypted?--but that's because it's well constrained within wires run by trusted people and such cost is not justifiable; but with the advent of cell phones, protocols like GSM had encryption built into them, since telcos couldn't ignore that data sent over the air is readily captured and manipulated.
Oh, I wouldn't say it took expert knowledge in the past. I've been able to download ettercap for years. It'll sniff switched networks. I just need to plug into your LAN.
>why isn't the POTS network encrypted?
Because I can't pick up the receiver and listen to my neighbor's calls. That's exactly how ethernet doesn't work. I hear all, even on switched networks with a simple tool. Wireless? Even worse, it's the equivalent of using a non-switched hub. I'll even toss in that POTS should be encrypted point to point, but if you would have attempted it before Phil Zimmerman trail-blazed strong encryption for us commoners, you would most likely have ended up in a federal prison if you used non-trivial encryption.
So, to address the original question:
1. Encryption really was expensive in 1996 or so, but in the age of multicores, its questionable if there's a real cost now.
2. Performance hit. SSL handshake is long and annoying. We could use a replacement here.
3. Legacy. The hackers who slapped together networks and the various protocols we use weren't interested or couldn't properly secure this stuff. Thus the legacy of spam, plaintext everything, etc. Retro fitting security is hard. Its good if its there from the start like GSM.
4. Standards are hard to change. Imagine telling all the browser makers and open source projects that we should all switch to something that won't add to the bottom lines/download rate/market penetration, but instead cost them time and complexity as well as add server load. Google can't get anyone to care about SPDY. Who is going to re-architect this stuff? We're essentially living in a world convcieved in the 70s, implemented in the 80s, and made popular in the 90s.
The easy answer is to stop allowing non-encrypted wifi. I'd love to see the wifi consortium just default to a random key for non-authenticated guests in whatever the newest version of wifi will be. It'll be transparent to the end user and probably stop all these local attacks. It won't help you past the gateway, but right now between the client and the gateway is the problem area. Heck, why not make all networking gear do this, even on wired networks? We have the processing power.
> 1. Encryption really was expensive in 1996 or so, but in the age of multicores, its questionable if there's a real cost now.
A lot of "budget-priced" hosting services are near-free for http and heavily-surcharged for https. - Pricing it to make it seem as if there were some huge cost difference. Makes a difference when shoestring clubs/nonprofits want to throw up a cheap static site.
That's because SSL is a shitty protocol, it has nothing to do with CPU.
Turns out that you can only have one SSL applied to one IP because browsers* can't tell the webserver hosting 1,000 sites which particular site you're requesting, because of, you guessed it, encryption.
Essentially, you're paying for a dedicated IP not CPU.
*theres a fix for this supported by several browsers, but it will never be backported to legacy browsers and the millions who won't upgrade for many years so its unsafe to assume you can use this method, thus one IP per SSL for the foreseeable future.
The problem is really only with IE on XP; of course, that's still a huge share. We can thank MS for that, since if other browsers support in on XP, there's no reason IE couldn't, especially considering 7+ already does in Vista and 7.
The problem with that is that systems like Skype require strong encryption to work at all, and Skype manages to provide high quality voice transmission despite using encryption on a wide variety of OSs and hardware.
It will probably be more technically difficult to make Skype friendly to snooping than it was to make it secure.
WiFi traffic has nothing to do with internet traffic per se. The problem is that the only popular WiFi protocol that is anonymous (doesn't require passwords/user credentials) is also not encrypted. This problem could elegantly be solved with a ssh-like tunnel between the consumer device and the WiFi router, the rest of the traffic could still be plain text.
The user's device would somehow have to be sure it's talking to the legitimate router, and not one an attacker set up with a deceptively similar SSID. I can't think of a way this would be possible, unless popular access points were signed by CAs.
This is the classic excuse to not implement encryption: "but we need a web of trust and certificate authorities and central management of identities and a million complex things that'll never work right even if implemented."
Encryption without identity validation is a worthy goal in itself, especially when we're talking consumer grade stuff.
That's really just another way of saying that users shouldn't connect to routers they do not trust.
There's this new feature of SSH key generators, where the user is shown an ASCII art of the generated SSH key, to be able to visually remember and recognize it. A standardized graphical representation of certificates could be used to confirm the identity of the router. Imagine walking into Starbucks or any WiFi enabled place, you see a sign on the wall that gives you the network name and the "VisiSign", you connect to the router, your computer shows you the VisiSign of the router, and you can visually confirm the identity of the router.
I think it's legitimate to want encryption even if you can't verify identity. Man in the middle may be trivial in that case, but it's still harder than just sniffing packets.
Only recently, with the spread of wireless networks, has the threat of en masse MITM and eavesdropping attacks become realistic. For a long time, it took serious knowledge to pull these things off on a wired LAN or within the walls of an ISP.
You could say the same about phone calls--why isn't the POTS network encrypted?--but that's because it's well constrained within wires run by trusted people and such cost is not justifiable; but with the advent of cell phones, protocols like GSM had encryption built into them, since telcos couldn't ignore that data sent over the air is readily captured and manipulated.