Here's hoping for next time.
This is simply false. It is untrue to claim that all U.S. companies have somehow "weakened" encryption or inserted backdoors in their products for the Feds. I normally wouldn't waste my time correcting conspiracy theories, but sometimes it's necessary to stop the more credulous from believing them.
Yes, the NSA has boasted of having a surveillance "partnership" with certain U.S. companies, but those are telecommunications carriers -- AT&T, Verizon, Sprint, etc., not Silicon Valley firms: http://www.cnet.com/news/surveillance-partnership-between-ns....
For an additional indictment of AT&T, look at the sworn affidavit that EFF obtained from local SF bay area whistleblower Mark Klein -- an AT&T technician who revealed the existence of the NSA's fiber taps at the 2nd & Folsom Street SF facility.
But the Silicon Valley companies that we know and more-or-less love have done the opposite. Look at the announcements about device encryption by Google and Apple in the last month (that have irked the Feds so much they're threatening new laws). Look at Google's Adam Langley, Wan-Teh Chang, Ben Laurie, and Elie Bursztein deploying a better TLS cipher suite in Chrome. Look at Twitter's surveillance lawsuit this week against the Feds over, apparently, the legality of a warrant canary.
And of course the two links in the conspiracy theory posted above prove the opposite of the "weaken encryption" claim. First, CALEA doesn't apply to web companies. And even the carriers it does apply to are permitted to (at 47 USC 1002(b)(3)) provide secure end-to-end encryption: "A telecommunications carrier shall not be responsible for decrypting, or ensuring the government’s ability to decrypt, any communication encrypted by a subscriber or customer, unless the encryption was provided by the carrier and the carrier possesses the information necessary to decrypt the communication."
Second, the FOIA'd doc was written in the late 1990s before the Feds liberalized encryption export controls. It's 15+ years out of date. You can now freely export strong crypto. And even in the dark days of the 1990s, there were no domestic controls on encryption use, though the TLAs did give it a shot at one point.
A better argument for the conspiracy theory set is the very odd relationship between EMC Corporation's RSA business unit and NSA. But even if allegations of intentional security flaws are true, EMC is a Massachusetts company, not a left coast firm, and a cozy relationship between the NSA and EMC/AT&T/VZ/etc. certainly does not indict all companies and their founders.
Finally Apple isn't a "web company", nor is Microsoft or the majority of 'large corporations'.
> "...unless the encryption was provided by the carrier and the carrier possesses the information necessary to decrypt the communication"
I don't think I need to say very much here.
Regarding Apple's encryption there's lots of good information about how very little it actually does and can do. There was also a hacker news thread with yours truly. https://news.ycombinator.com/item?id=8389365
Regarding the 'antiquated' FOIA doc - you can export strong crypto but you backdoor it, keep the keys, or provide other ways to access the data. Blackberry's entire business model was secure communications and look where they are today. RSA is now in a similar boat.
"Weaken encryption" does not mean 'lower the bit security under the standard attacker model' here. In this case it means 'subvert encryption through design or implementation flaws, key repositories, controlled PRNGs, side channel access or designed-in systems level access to the data'. "Weaken encryption" doesn't mean "choose smaller key sizes", it means "make the fact that something is encrypted a weak guarantee of security and privacy."
> But the Silicon Valley companies that we know and more-or-less love have done the opposite
Image management, nothing more. Why the public showdown? That doesn't make any sense at all. No sir, I'm certain that corporations would like to provide security and privacy for their customers. And they do. But not against federal law enforcement. There are no new laws that need passing right now. The machinery is there and we've witnessed it in action multiple times. There is no David and Goliath story here, no heroes to be heralded. It's romantic, alright. I wish it were realistic.
Google (and others, that we are supposed to love) fought several battles that made it to the highest levels of court in the United States but lost. They were then forced to comply. The United States used extreme financial leverage to get QWest to comply (and when they still wouldn't, let/forced them to go bankrupt).
What's changed since then? Where there some new Supreme or Circuit Court decisions that have depreciated the former?
Didn't we already chat about key escrow requirements, CALEA, etc?
The Clinton Administration's big thing was key escrow. Clipper, of course, and as that policy failed the administration moved control of encryption from Military/Exports and Munitions to the Department of Commerce under the agreement that key escrow systems would be put in place where weak cryptography had been previously. Major companies (including RSA, IBM, Apple, Sun, HP, AOL, others) collaboratively drafted industry standard key escrow systems (aside: crypto key escrow patents are a fun things to look up on patent searches).
Additional pressure was exerted of course because the United States had seen a very strong rise in geopolitical espionage and sabotage and strong crypto was becoming a problem for the NSA.
That's where things were at the end of the Clinton Administration. During Bush's administration we saw nothing but an expansion of powers and budgets for intelligence agencies and reclassification of laws applying to other media (for tap and trace/pen register) to the internet (although CALEA already applies to broadband internet) and to computers, 'computer systems and networks' and electronic equipment. Bush (and Clinton before him) warned of rising international cyberwarfare, but couldn't get the populace concerned about it. Anyway, you do NOT see a reversal on escrow requirements during the Bush or the Obama administration - rather you see an expansion of escrow and an expansion of hardware, software and standard backdoors as well as the leaks from Snowden.
There are a number of ways that escrow is done (we're ignoring backdoors right now). The TPM is one novel way that keys are stored in a way that gives access for law enforcement. TPMs are in essentially every computer, 'spooks' showed up at the standardization meetings for the chip, Germany announced they provided backdoor access during diplomatic troubles (and have since 'rescinded the announcement' whatever that means), China blocks all electronics with TPM chips coming from the United States (and allies) and after a bunch of international and technical/commercial problems the TPM 2.0 spec (again attended by Five Eyes spooks) it was for the most part abandoned. And honestly, does a low end consumer device ($650 laptop or $300 phone) require a self destructing chip that can't be examined using standard equipment or at room temperature? The TPM also almost always resides on the low pin count bus (the spec does not specify where it needs to sit), which gives it DMA (TPMs do NOT need DMA).
Or check out Microsoft's Cryptographic Service Provider (CSPs). This is where you store, provision, can generate, access, and utilize your keys in a Microsoft system. It's also famous for being where the NSAKEY gave access. Microsoft will tell you that it keeps your keys secured, but it is well known to any pentester that access to admin on a box can dump all of the keys, including those that are so-called 'marked non-exportable'. From what I can tell by the public MSDN articles on the subject contents of the CSPs can be controlled via group policies, and interops with other management systems.
This isn't to mention that bitlocker keys are automatically synchronized with Skydrive (Onedrive) accounts and that Onedrive was onboarded to PRISM for NSA access. Well, that's only if you have a Microsoft Account. Oh, one is automatically made for you and you're essentially required to sign up for an account to use any new Microsoft OS.
Well, that and bitlocker keys are also backed up inside of organizations to Active Directory (i.e. don't 'domain join' your personal computer).
Check the flurry of Apple news items recently. The narrative would like to use that as some sort of David vs. Goliath story of the good guy capitalist protecting his consumer. But check the details. The addition of encryption is not new. What's new is that they are publicly claiming that with this encryption system they will not have access to the private keys, and so can not comply with requests. Now we don't have to believe them (I don't), but we do have to acknowledge that -not having a facility for escrow- is their 'dangerous game'. Encryption is not a 'dangerous game'. Not providing escrow is.
You're right that it's unreasonable to place blind trust in a closed encryption system, even Apple's, that we're unable to review or audit. Even if their intentions are pure, it could be poorly implemented.
But my point is simply this: there is no U.S. law mandating key escrow for Apple, Google, Microsoft, etc. One was proposed in the 1990s. It didn't pass. One was proposed in the Going Dark era 4-5 years ago. It didn't pass. One is being quasi-proposed now. It hasn't passed.
I actually think I'll agree with you on a lot of issues based on your post above -- but it is nevertheless a conspiracy theory to claim that Silicon Valley companies somehow engage in key escrow for the NSA or that there is a legal requirement for them to do so. As I said before, if you claim otherwise, URL, please.
>NSA currently requires companies (like Microsoft, Apple) to provide encryption keys corresponding to devices
PS: ^^^ I'm still waiting for the link that backs up this claim too.
However, this is not to discount law in praxis, how CALEA is interpreted and enforced, the history of key escrow, current technology considerations, known and suspected escrow mechanisms, and pressures exerted by federal law enforcement (e.g. the removal of effective crypto from Skype when it captured its market), and how all of this hangs together.
My (reasonably, educated and technically informed :p) suspicion is that companies at a certain size, federal agents will come to your company and make demands for data, which you must comply with and slowly as you are compelled by law to give keys and data on a regular basis it becomes the best thing for your business to install automatic escrow mechanisms.
It is my assertion that law enforcement interprets laws that read as "...unless the encryption was provided by the carrier and the carrier possesses the information necessary to decrypt the communication" as enough to force companies to create escrow mechanisms.
I would argue some of this story played out very publicly with Google. Another great example here would be the NSL of Lavabit - how they asked for far more than was legally obligated and the forcefulness and non-public nature of the demands made it impossible to put forward a reasonable defense.
To summarize it is apparent to me that the difference in our perspectives is whether a specific law (another being proposed again now, as you point out) is required in the current climate of practice of law, along with the leverage and the compliance requirements that exist inside it, for key escrow to be 'required of companies'.
I come down on the side of 'no'. I believe they have what they need now to get escrow.
Maybe this Apple and Google thing will clarify it. But somehow I doubt it. Prediction: no explicit law on key escrow will be passed (it would be entirely too harmful for US exports) but it will continue to be practiced.
HP obtained a patented backdoor system in 2008 (filed mid 2000 and approved around the start of 2001).
Certicom (the same company DUAL_EC was filed under) has a new key escrow system patented.
Key escrow system filed in 2004 and published in 2011:
2008-2012 stateless key escrow patent:
"Automatic recovery of TPM keys" - Lenovo
Oops, here's a Microsoft "cloud storage" key escrow system from 2011-2012.
"Some of the files stored on the network server may be sensitive or confidential. The user may wish to restrict access to those files. The files may then be encrypted or otherwise protected with a password or other key. The user may not trust the network server to store the key, and may thus desire to retain sole possession of the key. Such users, however, often lose (or forget) their passwords or keys. Moreover, other third party users may legitimately require access to the stored, encrypted files."
"FIG. 3 illustrates a flowchart of an example method for providing third party data access to a user's encrypted data according to a predefined policy."
"In some cases, however, while the data storage system is not able to decrypt the user's data, it may be necessary for an outside entity (e.g. a governmental entity) to access the user's data."
"In cases where the encrypted data is a cryptographic key, that key may be stored as a plurality of shares. The shares are mathematical transformations of the user's private key, and each share is provided to one of the verified third parties. Each verified third party publishes his or her own public keys, and encrypts his or her share of the encrypted key using their published public key. The verified third party shares encrypted according to the third partys' public keys are then stored in the data storage system. Because the shares are encrypted according to the verified third parties' public/private key pair, the data storage system is prevented from accessing the encrypted shares, and is further prevented from accessing the user's data."
"In some cases, the user's data may comprise, at least in part, a cryptographic key. The user's data, including the key, may be stored in multiple different shares."
Encrypted data AND keys!
The patent examiner cited "The Risks of Key Recovery, Key Escrow, and Trusted Third-Party Encryption" when doing his examination.
Some Raytheon for good measure. A bit older, but certainly Bush era.
Motorola . Certicom . Honeywell . Motorola . Deutsche Telekom Ag . IBM . Apple . Fujitsu . Red hat . F-Secure . Sony . Seagate . Dell . Sprint . Nokia . Gemalto . Intel . Samsung . Symantec . Sun Microsystems . Toshiba . Cinea . General Dynamics . Citicorp . Siemens . Ericsson . VMWare . Facebook . HP . Cisco . Liquid Machines . GIC . Microsoft . Novell .
And of course the US patent system has a classification for key escrow systems: 380/286.
 https://www.google.com/patents/US8195959 & https://www.google.com/patents/US7856664 & https://www.google.com/patents/US7873170
 https://www.google.com/patents/US6754349 & https://www.google.com/patents/US8055911 & https://www.google.com/patents/US7752318
 https://www.google.com/patents/US8144876 & https://www.google.com/patents/US8494169 & https://www.google.com/patents/US20070280483
 https://www.google.com/patents/US8315386 & https://www.google.com/patents/US20070172069 & https://www.google.com/patents/US7492895
 https://www.google.com/patents/US7050589 & https://www.google.com/patents/US7660423 & https://www.google.com/patents/US20100142713
There are circumstances where this is not being used to give access to law enforcement, or the patent would not make sense in that context.
This is a last defense argument. Certainly it is the case for some escrow systems, and (especially with the patentese) are difficult to decipher.
But others are clear as day: "In order to receive the information, law enforcement may submit a request to each of the entities identifying the communication session and their basis for authorization."
Only some of the documents provided above have been read through by yours truly. They constitute only a minor fraction of patents by a minor fractions of companies in the US patent system.