Just your regular reminder that for the only security certification that Apple advertises on their website for iOS [1][2] Apple only achieved the lowest possible level of security assurance, EAL1. A level only fit for products where [3]: "some confidence in the correct operation is required, but the threats to security are not viewed as serious" which does not even require "demonstrating resistance to penetration attackers with a basic attack potential" [4]. This is four entire levels lower than "demonstrating resistance to penetration attackers with a moderate attack potential" [5].
Apple has never once, over multiple decades of failed attempts, demonstrated "resistance to penetration attackers with a moderate attack potential" for any of their products. To be fair, neither has Microsoft, Google, Amazon, Cisco, Crowdstrike, etc. It should be no surprise that the companies, processes, and people who lack the ability, knowledge, and experience to make systems resistant to moderate attackers despite nearly unlimited resources are regularly defeated by moderate attacks like commercial surveillance companies. They certify that they absolutely, 100% can not.
Please don't post "regular reminder" style comments - they're too generic, and generic discussion is consistently less interesting. Good threads require being unpredictable. The best way to get that is to respond to specific new information in an article.
> neither has Microsoft, Google, Amazon, Cisco, Crowdstrike, etc. It should be no surprise that the companies, processes, and people who lack the ability, knowledge, and experience to make systems resistant to moderate attackers
Companies create a separate SKU for products that meet higher levels of security assurance for Common Criteria. I know for a fact that the companies you listed offer SKUs that meet higher EA levels (EAL4+) for Common Criteria. You just gotta pay more and purchase via the relevant Systems Integrators.
A consumer product line like the Apple iPhone isn't targeting DoD buyers. That's always been Blackberry Ltd's bread and butter
I said, "resistance to penetration attackers with a moderate attack potential". EAL5 is the first level at which you must demonstrate that as can be seen in my 5th link [1] which bolds the diffs from the previous level.
None of those companies has ever once certified a product to that level as far as I am aware. The failure is so complete that it is generally viewed as impossible to fix the structural defects in products that failed a EAL5 certification without a total rewrite. It used to say that in the standard somewhere, but the standard revisions have moved it so I can not quote it directly.
No. The US government briefly had procurement requirements for high security deployments.
They were forced to relax them because Microsoft could not make bids that met the minimum requirements for DoD and high security projects and that made their Senators mad. They relaxed them to EAL4+ because that was the most that Microsoft could do.
They since relaxed them further to EAL2 because that is all the most large AV and cybersecurity appliance vendors could achieve. They justified it under the "swiss cheese" model where if you stack multiple EAL2 then you get EAL4 overall, which is insane. The government has since relaxed them even further since none of the companies want to do any certification since none of them can achieve a decisive edge over the others that they can write into the requirements thus disqualifying their competition, so certification is just a zero-sum game.
EAL4+ is useless against the prevailing threat actors as can be seen time and time again. There is no point at aiming for inadequate; even if you get there you still get nothing.
EAL6-7 certifications are basically the only known, existing certifications that have any evidence supporting that they are adequate to defend against the known and expected threats. As far as I am aware, there are no other certifications even able to distinguish products that can viably protect against organized crime and commercial spyware companies. Existing products max out every other certification and we know for a fact those products are ineffective against these threat actors. Therefore, we can conclude that those certifications are useless for identifying actual high security products adequate for the prevailing threat landscape.
Sure, if we had some other certification that could certify at that level and was more direct, that would be nice. But we do not, the only ones that we know to work and that products have been certified against are Common Criteria EAL6-7 (and maybe EAL5). We can either choose certifications that are cheap and do not work, or ones that work. Then, from the ones that work, we can maybe relax the requirements carefully to identify useful intermediate levels, or identify if some of the requirements are excessive and unnecessary for achieving the desired level of assurance.
However, the key takeaway from this is not whether we can certify products to EAL5 and higher or whether those certifications work or the cost-benefit of that certification process. The key takeaway is that EAL4 is certainly inadequate. Any product in commercial use targeting that level or lower is doomed to be useless against the threat actors who we know will attack it.
The AVA_VAN (vulnerability analysis) Security Assurance Requirement (SAR). AVA_VAN.4 requires “resistance to penetration attackers with a moderate attack potential”. AVA_VAN.4 is only required for EAL5 and higher.
You could individually incorporate a higher AVA_VAN into a lower EAL as a augmentation, but few do that. You also do not get any of the other conformance assurances that a higher EAL gives you. There is a reason we use EAL as a whole instead of just quoting the AVA_VAN at each other.
Though maybe you are talking about the Security Functional Requirements (SFR) which define the security properties of your system? That is somewhat orthogonal. You have properties and assurance you conform to the properties. Conformance more closely maps to “level of security” as seen in the AVA_VAN SAR. However, the properties are just as important for the usage of the final product because you might be proving you absolutely certainly do nothing useful.
I feel like you're arguing that these certifications are useless and uncorrelated with security but then you're trying to say that Apple and others are bad for not having them.
Low certification levels certify low levels of security. High certification levels certify high levels of security.
EAL4 is known to be too low against modern threats that will attack commercial users. We know this from experience where EAL4 systems are routinely defeated. Higher certification levels, such as the SKPP at EAL6/7, are known to be able to resist against much harder threats such as state actors like the NSA (defeating a NSA penetration test was a explicit requirement tested in the SKPP by the NSA themselves).
Low certification levels, like EAL4 and lower, that are the limit of the abilities of companies such as Apple and Microsoft are known to be useless against commercial threats. They are uncorrelated with protection against commercial threats because they are inadequate in much the same way that having a piece of paper in front of you is uncorrelated with surviving a gunshot. Systems that can only be certified to EAL4 and lower are certifiably useless.
> Low certification levels certify low levels of security. High certification levels certify high levels of security.
I guess I don't know enough to say but I just doubt that, knowing what I know about other certifications. I expect that they're perhaps lightly correlated with security.
You said that I was arguing the certification is useless. I was arguing that certifying to low levels is useless. Those are not even close to the same argument.
For example, a squat test is a reasonable measure of leg strength. Only squatting 20 kg means your leg strength is extremely weak. The test procedure is fine, getting results like that is not. If that is all you can do, that is quite problematic.
As to the certification itself, it is pretty good. Easily hacked products like iOS, Linux, and Windows are consistently unable to certify as moderately secure. That is vastly different than basically every other certification where products like Windows pass with flying colors even though we all know that is nonsense.
So, at the very least, low certification levels like EAL4 provide high confidence of lackluster security. You can withhold judgement of high assurance levels corresponding to high security if you like, but low assurance levels corresponding to low security is pretty clearly established.
I mean, it occurs to me that maybe all of these companies aren't doing this for a reason - because common criteria and compliance are often stupid and don't represent real security. Perhaps these policies are the exception? But I've managed SOC2 for example and I can definitely say that there are plenty of ways to get your SOC2 without giving a shit about actual security.
They failed. Repeatedly. For decades. They spent billions trying. The failed so much that the standard writers determined the only logical conclusion is that it must be practically impossible to retrofit a system that failed EAL5 certification to achieve EAL5 or higher certification without a complete rewrite and redesign. It says so right there in the standard [1]: "EAL4 is the highest level at which it is likely to be economically feasible to retrofit to an existing product line". That was added due to the decades of experience where everybody who ever tried to do that failed no matter how much time or money they spent.
We also have plenty of evidence that it does matter, they just can not do it. Here is Google touting their Common Criteria certification for the Titan M2 security chip hardware which is EAL4 + AVA_VAN.5 (resistance against penetration attackers with a high attack potential) [2]. Note that this is only the hardware (software was not certified; a critical severity vulnerability was actually disclosed in the software allowing complete takeover if I remember correctly) and only cherry picks AVA_VAN.5 so is still only EAL4, not a holistic EAL6 certification. Getting that certification was a deliberate effort and cost. If they literally did not care about the Common Criteria then they would just certify to the checkbox level like everybody else. It is because they could certify it to a higher level than most other can achieve that they chose to do it because then they could tout their unique advantage.
Basically everybody gets a certification and basically everybody displays their certification on their page. There is something to be said about them opting for a EAL1 over a EAL4. It is basically assumed that any serious vendor could probably get a EAL4 with some effort. So, there is no differential advantage to displaying a EAL4 since everybody could get it. It is just a zero-sum game to pay for certification if everybody knows nobody has a true advantage. However, if you can achieve EAL5 or higher, then you do have a unique advantage because basically nobody else can do it. The fact that none of the major vendors attempts EAL5, shows that they can not do it.
> Apple has never once, over multiple decades of failed attempts, demonstrated "resistance to penetration attackers with a moderate attack potential" for any of their products. To be fair, neither has Microsoft, Google, Amazon, Cisco, Crowdstrike, etc.
So, OK I guess?
It's worth noting that CC evaluation does not score the actual practical security of a device or system, but the level of testing it was submitted to, which is consistent with pretty much every single governmental certification out there.
Sure it does, it is just that EAL4+, the highest level any of them can reach, does not certify "resistance to penetration attackers with a moderate attack potential". Guess what, commercial hackers have "moderate attack potential".
You are complaining that the 40 cm high jump test does not score actual jumping ability. You are right, it is a low bar that they should all be able to pass. You can not use the 40 cm high jump test to distinguish them. What you need to do is use the 100 cm high jump test. Some can pass it, but none of the large commercial vendors can. Sure, it would be nice if we had more gradations like the 60 cm and 80 cm tests, but we do not really know how to do that, so the best we can do is the 100 cm test.
I'm not really complaining, though. I'm just saying that security certifications are more about compliance than actual proof that a system cannot be easily compromised. In other words, they are more about legal requirements than guarantees.
It is also misleading to assert that a device or a system are less secure because they haven't been certified. Vendors submit requests to validate against specific levels or certifications, and it is not the goal of the certification authority to determine "how high" they score.
They can not certify against useful assurance levels. They have tried repeatedly for decades and spent huge gobs of money. It is not a choice, they are incapable of it.
I am judging them by their maximum ability ever demonstrated under the most favorable circumstances and they still can certify resistance against moderate attackers. They have never developed systems that can protect against the prevailing threat landscape and they can not develop such systems. Their best is not good enough.
> They have tried repeatedly for decades and spent huge gobs of money. It is not a choice, they are incapable of it.
First of all, I don't think that's true, and if it is, I would like to see proof of Apple submitting their products for evaluation.
Second, you are judging an entire industry. This is not about Apple shortcomings, there isn't a single vendor doing what you say needs to be done.
Regardless, and this is more a personal opinion than a hard fact, most certifications out there are BS. PCI-DSS is basically a checklist of best practices, CC goes from common sense stuff to essentially impossible to achieve unless designed for the specific purpose, etc. Yes, all these helped create a very healthy - and profitable - industry where consultants have thrived on Powerpoints and PDFs, without really creating any tangible value.
Yup. Want to have a laugh? Here is the Apple iOS certification report [1].
On PDF page 26 (document page 21) they describe the rigorous AVA_VAN.1 vulnerability analysis certification process they faced. The evaluation team basically typed in: "ios vulnerabilities" into Google and then typed in "ios iphone" into the NVD and verified that all of the search results were fixed. AVA_VAN.1 certification please.
To explain why, AVA_VAN.1 does not require a independent security analysis, it only requires a survey of the public domain for known vulnerabilities [2]. You need AVA_VAN.2 (which is only required in EAL2 and EAL3) before they actually attempt to look at for vulnerabilities themselves.
Common Criteria EALs have nothing whatsoever to do with practical security. I'd be surprised to hear anybody on this site who has managed a CCTL security review for a product saying anything positive about the program.
A fun exercise: find a list of commercial mainstream products with "high" EAL audits, and then look at their vulnerability histories.
The archetypical EAL5 product is a smartcard or cryptographic coprocessor (same thing, different package). They're certifiable because they don't do much.
But if you'd like an example from the EAL4 list: start with FortiOS.
EAL4 and under is junk (with respect to security). I already said that. The standards committee has also always maintained that there is no meaningful security at EAL4. Earlier drafts of the standards said EAL4 is only meant to protect against “casual and inadvertent attacks”.
The general crappiness of EAL4 goes all the way back to the Orange Book where EAL4 maps approximately to Level C2 (contemporaneous projects got certified to those levels simultaneously). That level was intentionally meant for toys before they put on their big person pants and release a grown up product with actual security [1]. It is a mystery why anybody thinks EAL4 and security belong in the same sentence.
If you're saying "EAL4 and below is junk" (I'd say EAL* is junk, but whatever), all you're really saying is that minimal-function cryptographic coprocessors are safer than operating systems and applications. Well, yeah, sure. I don't think you needed the farce of Common Criteria to tell you that, though.
But upthread, you knocked Apple for not achieving an adequate assurance level. As you can see now, and from your own last comment, that doesn't make any sense. It's possible (though deeply silly) that there's some iPhone configuration that could "achieve" EAL4, but you yourself don't believe that has any meaning. I don't either.
I don't think EAL5 or EAL6 do, either, except that if you tell me your product is EAL5, I'll assume it's a small fixed-function device.
No. The Separation Kernel Protection Profile (SKPP) defines a model for a operating system kernel at EAL6+. So all I am really saying is that safe operating systems are safer than non-safe operating systems. You could also look backwards to the TCSEC and the comparable certified Level A1 systems for other operating systems designed for actual high security work.
You keep calling it a farce, but you keep pointing at EAL4 and lower systems. Yes, those levels are farces, that was the whole point. Those are the levels for the certification of toys where documentation and paperwork is all that is needed, not proper design.
Complaining about the Common Criteria in the context of EAL4 and lower systems is like complaining about tissue paper manufacturers putting their tissue paper through bulletproof vest testing and certifying that it does not stop bullets. Yes, that is pretty stupid, farcical, and probably a waste of time. But no, the test is not stupid. It can test actual bulletproof vests, you just keep seeing stupid waste of time tests proving a useless fact that everybody already knows, the EAL4 quality system sucks and has no place in a serious security organization.
There's probably better/longer, I just looked at the first page of "standards" search. The points are strong, like messageboard-argument-winning bear, though.
What'd I do here? You have me worried. Did I manage to back into an argument that standards processes are good? Really I'm just trying to say two things here:
* The Common Criteria process is a farce, and things that are secure by dint of being EAL5+ are really secure because they're so small you can almost prove them with formal methods (which is not to say that everything EAL5+ has usefully applied formal methods).
* The meaning of the word "operating system" is different when we're talking about EAL5+ stuff; seL4 is an "operating system" that has been proven secure with formal methods, and it is secure, but it's secure in large part because it does almost nothing; it isn't comparable to Windows, macOS, or Linux. You couldn't build an iPhone experience on top of it (you could --- and people do --- host Unix kernels on top of L4 OS's, but when you do that you keep many/most of the security issues of that Unix kernel).
No no, sorry, it wasn't intended as backhanded snark.
You've had a running critique of standards stuff here for years and it's good and the other person should read it. A bit of an unsuppressed "well, maybe if they looked at the principles, then they'll see the light" nerd-response on my part. You're not going to make an iPhone out of seL4, you're also not going to come up with a Secure Enclave or an IOMMU or an encrypted serial bus or whatever by reading standards/meeting certification reqs either because, generally, that's not how standards and certifications work. For multiple developed versions of this argument, see 'tptacek!
I'm sure you're right about the details of Jor-EAL5+ and the bottle city CRYSTALS-Kandor, just saying "also right in this other way but with less lore". Not that there is anything wrong with lore.
So what you are saying is that EAL >=5 does not imply secure, it is the formal methods that imply secure?
Well good news, that is not a mistake. EAL >=5 implies formal methods of various degrees. The standard demands the thing that works, amazing. It might be too hard for consumer companies to achieve, but that does not make the process a farce as long as there are companies that can certify useful functionality according to the process. And, although you seem to think the functionality that can be certified is limited, I doubt you consider secure cryptographic co-processors useless or unimportant.
In that paragraph, the words "of various degrees" is load-bearing.
I don't think you're making a coherent case for the security or insecurity of an iPhone with any of this stuff. Real security in complex products simply has nothing to do with the CCTL process. Which, I'm sorry to say again, is farcical.
If it's not clear: I've had the displeasure of working with this process in my career, most of which I've spent in vulnerability research.
Yes, various degrees; we are discussing three distinct levels. Why on earth would they all demand the same degree of formal methods?
Up to EAL4 all you need is a informal specification; what is basically useless paperwork.
It is only at EAL5 that you are required to supply a semi-formal design, ADV_TDS.4, and interface specification, ADV_FSP.5. At EAL6 you must also supply a formal model of the security policy, ADV_SPM.1, and a complete mapping between design and implementation, ADV_IMP.2. By EAL7 you are required to supply a complete formal design, ADV_TDS.6, and specification, ADV_FSP.6.
You then have a formal model of the security policy, the interface enforcing that policy, the design backing the interface, and a mapping from the formal design to the implementation. That is pretty dang exhaustive. I guess you could go further and demand a formal, machine-checkable correspondence between the implementation and the design?
Since you bring up that you have worked with the process, what products at EAL5 or higher have you had the displeasure of working on?
This thread keeps trying to escape down little rabbitholes of abstraction. I'll be clear: if Apple wanted to ship an EAL5 product, the very first product decision they would need to make in service of that would be to stop rendering HTML. No browsers. No rich media. No installable apps that could do any kind of IPC.
This is what we mean when we say an EAL5+ product is a different kind of thing. It's why this whole CCTL EAL thing is such a farce. The methods you're blaming big tech companies for not using are all things that preclude most of the products users want to use.
Can we stop pretending that any vendor in the industry has the option of serving customers with formally verified "EAL6" products? They don't. This isn't a thing, and hearing "EAL6" and "penetration testing" mentioned in the same sentence is grating. You don't penetration test a formally verified product. You do verification and assurance work for it, but nobody in the field would ever call it pentesting.
I am blaming them for selling products that are inadequate for the threat environment they are expected to operate in and lying and/or insinuating that they are adequate for that threat environment, especially when they know for certain that they are not and certify as such. If the customers truly want those products and features, security be damned, like you say then they will do that even if the companies are completely truthful. The companies do not because they know that it will hurt their margins or make their products non-viable.
In addition, the lies suck all of the air out of the room for actual secure products because why go through the extremely hard work of actually making something secure when you can just lie about it. This has happened time and time again. There were multiple TCSEC Level A1 certified implementations when the DoD demanded real security. But, as soon as they lowered requirements to allow consumer systems that were inadequate for the threat environment, all of the effort and funding behind actual secure systems dried up. What we are left with now is a total wasteland of insecure products controlling systems too big for their britches and endless attacks getting closer and closer to total societal disruption. The only saving grace is that it is taking time for the attackers to scale to exploit this entire greenfield opportunity.
As to your other points, the SKPP explicitly included a penetration test done by a NSA team of the formally verified product under certification [1]: "AVA_VLA_EXP.4.3E The NSA evaluator shall perform independent penetration testing.". So, actually, "EAL6" and "penetration testing" do belong in the same sentence.
If Apple did want to ship a EAL5 product, the very first product decision would be starting over. HTML support or not does not even figure into the list; they have to tackle much more basic defects before getting to that. And I do not see how HTML and rich media support is even a challenge unless you made your security properties depend on exact rendering. You just use a separation kernel architecture to isolate the untrusted HTML renderer into a ephemeral sandbox that takes a HTML file and outputs an image. You might then say, "Apple already sandboxes the renderer.". Yeah, but their sandbox sucks and is regularly defeated. Almost as if having formally verified separation kernels as an underpinning might allow the obvious solution to work; assuming the same minds that built on sand do not add in more harebrained ideas. Hard to put it past them when they made so many other terrible security decisions.
Also, you did not answer what EAL5 or higher products you worked on that informed your disgust with the high assurance Common Criteria process.
I am blaming them for selling products that are inadequate for the threat environment they are expected to operate in and lying and/or insinuating that they are adequate for that threat environment
You can make this argument and remove all the Jor-EAL5 stuff after and it's the exact same argument, in fact, a better one because you don't have all that superfluous stuff. It's a parasitic red herring.
> I am blaming them for selling products that are inadequate for the threat environment they are expected to operate in and lying and/or insinuating that they are adequate
You lost me here. Where has Apple "insinuated" anything else but "secure enough for consumers"? Really, I want to know where they promote their products as the choice to protect oneself from nation-state adversaries, because so far, the only security offers they have for iPhone users are threat notifications [0] and lockdown mode [1], and they make no guarantees on either of them.
Samsung [2] and Google [3] make similar assertions.
I believe you are targeting a strawman here, especially given the fact that there are zero consumer grade phones out there that are CC certified to a level that you may consider adequate.
> In addition, the lies suck all of the air out of the room for actual secure products because why go through the extremely hard work of actually making something secure when you can just lie about it.
This is blatantly false.
There aren't consumer ready operating systems that may replace Apple's, and are EAL5+ certified.
Unless your point is that some day you may be able to install System Z in your laptop, that is.
Do you by any chance have this data on Google, Samsung, Huawei, LG, and other cell phone manufacturers? I’ve never looked into these certifications and I wouldn’t know where to start looking. Do the above companies publish the results like Apple?
Generally only components are EAL certified. For example, the iPhone is not on there, but the security protecting access to Apple Pay on the iPhone 13 with A15 Bionic running iOS 15.4.1 (19E258) is EAL2+.
Sure. The Common Criteria for Information Technology Security Evaluation [1] is the foremost internationally recognized standard (ISO 15408) for software security that most large companies certify against for at least some of their product portfolio. I believe there are US government procurement requirements to that effect, so many systems will have certifications of some form.
For many companies you just search: "{Product} Common Criteria" and they will usually have a page for it on their website somewhere.
For smartphones you can see them there under "Mobility".
Unfortunately, it is fairly hard to parse if you are not familiar with the terminology. The general structure of Common Criteria certifications is Security Functional Requirements (SFR) which are basically the specification of what the product is supposed to do and the Security Assurance Requirements (SAR) which are basically how you certify the SFRs are met (and what level of assurance you can have that the SFRs are met). SARs can be bundled into Evaluation Assurance Levels (EAL) which define collections that reasonably map to levels of confidence. You can add SARs beyond the current EAL which is how you get a EAL level with a +, but it is important to keep in mind that just cherry picking certain SARs does not necessarily give you a holistic assurance improvement.
SARs and SFRs can be further pre-bundled into Protection Profiles (PP) which basically exist to provide pre-defined requirements and testing methodologies instead of doing it one-off every time. Some Protection Profiles support variable EAL/SAR levels, but these days people generally just certify against a Protection Profile with a fixed SAR bundle. This is what PP Compliant means. If you want to see what they certified against, you would need to look at the Protection Profile itself.
For smartphones, the standard Protection Profile for the phone itself is Mobile Device Fundamentals. If you look at the SAR bundle there you will see that they correspond to EAL1 + a small number of EAL2, resulting in a overall level of EAL1+. As they are in-between EAL1 and EAL2 I just classified it as EAL1 for my earlier post. If you peruse further you will see that basically every Protection Profile that companies certify to as PP Compliant are basically the same EAL1+ or thereabouts. So, if you see PP Compliant, it probably means EAL1+ or so.
Apple has never once, over multiple decades of failed attempts, demonstrated "resistance to penetration attackers with a moderate attack potential" for any of their products. To be fair, neither has Microsoft, Google, Amazon, Cisco, Crowdstrike, etc. It should be no surprise that the companies, processes, and people who lack the ability, knowledge, and experience to make systems resistant to moderate attackers despite nearly unlimited resources are regularly defeated by moderate attacks like commercial surveillance companies. They certify that they absolutely, 100% can not.
[1] https://support.apple.com/guide/certifications/ios-security-...
[2] https://support.apple.com/library/APPLE/APPLECARE_ALLGEOS/CE...
[3] https://www.commoncriteriaportal.org/files/ccfiles/CC2022PAR... Page 14
[4] https://www.commoncriteriaportal.org/files/ccfiles/CC2022PAR... Page 16
[5] https://www.commoncriteriaportal.org/files/ccfiles/CC2022PAR... Page 20