With that in mind, it'd be handy to know which exploit techniques these steps break, and whether those steps are in the current "meta" game for exploit developers.
(The specific mitigation here: the kernel formerly locked system call invocation down to the libc.so area of program text in memory; libc.so is big, so now OpenBSD locks specific system calls down to their specified libc stubs; further, in static binaries, the same mechanism locks programs down to only those system calls used in the binary, which effectively disables all the system calls not explicitly invoked by the program text of a static binary).
Indeed, in CCC's "systematic evaluation of OpenBSD's mitigations"[0] the presenter explicitly calls out OpenBSD's tendency to present mitigations without specific examples of CVEs it defeats or exploit techniques the mitigations are known to defend against:
> Proper mitigations I think stem from proper design and threat modeling. Strong, reality-based statements like "this kills these vulnerabilities," or "this kills this CVE; it delays production of an exploit by one week." And also thorough testing by seasoned exploit writers. Anything else is relying on pure luck, superstition, and wishful thinking.
Some of OpenBSD's mitigations are excellent and robust in defensiveness; others are amorphous and not particularly useful.
> Proper mitigations I think stem from proper design and threat modeling. Strong, reality-based statements like "this kills these vulnerabilities," or "this kills this CVE; it delays production of an exploit by one week." And also thorough testing by seasoned exploit writers. Anything else is relying on pure luck, superstition, and wishful thinking.
The comment seems to imply that "proper design and threat modeling" must stem from real-world CVE-s and proofs of concept. That seems to me like "if nobody heard it, the tree didn't fall" kind of thinking.
I'm sure OpenBSD developers have very good intuition on what could be used in a vulnerability, without having to write one themselves. And fortunately, they don't have a manager above them to whom they need to justify their billing hours.
>I'm sure OpenBSD developers have very good intuition on what could be used in a vulnerability, without having to write one themselves
Why? On average programmers are not very good security engineers. And the opposite - security engineers are often not a good programmers. If your mitigation doesn't stop any CVE that's being exploited right now in the wild, it's an academic exercise and not particularly useful IMO.
>And fortunately, they don't have a manager above them to whom they need to justify their billing hours.
The point of the thread is that the mitigation cost right now may be low (the "billing hours"), but it's paid in perpetuity by everyone else downstream - in complexity, performance, unexpected bugs, etc. So having a manager or BDFL to evaluate the tradeoffs may be beneficial.
> If your mitigation doesn't stop any CVE that's being exploited right now in the wild, it's an academic exercise and not particularly useful IMO.
If your only metric of security is "fixed CVEs", then you're rewarding mistakes that were rectified later, and punishing proactive approach to security that actually makes fewer CVEs appear in the first place.
And Theo's reputation and influence on the security is evidence that what he does is more than just "academic exercise".
E.g. he created OpenSSH.
> The point of the thread is that the mitigation cost right now may be low (the "billing hours"), but it's paid in perpetuity by everyone else downstream - in complexity, performance, unexpected bugs, etc.
While that may or may not be the pattern in general, it is not a rule, and especially doesn't apply in OpenBSD development. OpenBSD is widely regarded as one of the cleanest and most robust (free software) codebases ever.
You're mischaracterizing their logic. They're saying it's a necessary but not sufficient metric. You can't then shoot it down for being not-sufficient; we all agree about that.
It's not my recollection that Theo created OpenSSH, for what it's worth. My memory of this is that it was mostly Niels and Markus who did the lifting.
You might do some digging on Theo's reputation among exploit developers. It's complicated.
> They're saying it's a necessary but not sufficient metric.
Okay, then I'm saying it shouldn't be necessary either, for the sole reason that preventing a future CVE is not measurable, while fixing a CVE is. If you so much as pay attention to fixing existing real-world CVEs, you're implicitly focusing on that measurement, as you cannot predict the future. I argue that we would be better off not paying attention to them at all.
If anything, we should take the wide array of CVEs that were discovered in other systems and not applicable to OpenBSD as evidence that their intuition and proactive approach works well. The only real metric of a security of a system is the absolute number of CVEs in a long period of time, in which OpenBSD shines.
>> I'm sure OpenBSD developers have very good intuition on what could be used in a vulnerability, without having to write one themselves
> Why?
Exactly, POCOGTFO! :)
But wouldn't providing such a proof-of-concept implementation immediately render a bull's eye on all pre -current (and/or not appropriately syspatched) boxes in the wild?
They famously do not. That's OK, it's a trait shared by a lot of hardening developers on other platforms, too --- all of them are better at this than I'll ever be. But the gulf of practical know-how between OS developers and exploit developers has been for something like 2 decades now a continuing source of comedy. Search Twitter for "trapsled", or "RETGUARD", for instance.
> But the gulf of practical know-how between OS developers and exploit developers has been for something like 2 decades now a continuing source of comedy
Are you implying that OS developers are 2 decades behind exploit developers? If so, is there any proof of that claim, e.g. OpenBSD exploits?
Or are you implying that OS developers are 2 decades ahead of exploit developers? If so, how is that a bad thing?
Neither, I'm saying that for the past 2 decades, the conventional wisdom in the space has been that OS hardening efforts were some significant quantum of time behind exploit developers, but certainly not "2 decades" worth.
It's an aggregate sentiment, right? There are some mitigations that I think legitimately did set back exploit development, but on the whole I think the sentiment has been that OS hardening mitigations have been not just reactive, but reactive to exploit development that is some significant quantum of time behind the current state of the art.
By way of example, I think people made fun of the original OpenBSD system call mitigation stuff described at the beginning of this post. I have no idea what the consensus would be on this new iteration of the idea.
I'll bet the NSA is very happy about this situation and is doing everything they can to keep the gravy train rolling.
I thought the entire point of being a good security person was that you're able to anticipate and defend against attacks before they become known... Isn't that what "security mindset" is supposed to entail?
NSA doesn't care about your emailed vulnerability report. They're not spending their own money when they buy zero-day bug chains in platforms people actually use, and even if they were, those bug chains are so ludicrously cheap relative to their utility that any sigint (or law enforcement, for that matter) organization in the world, from Canada to El Salvador, can cheerfully afford them.
Even if your emailed report was a complete bug chain and not, like, an X-Frame-Options redressing issue, it would be harder, and probably more expensive, for NSA to pick the bug up from email than it would be for them to simply fill out a purchase order from one of their private partners.
As always it is helpful to remember as well that NSA's mission is to secure budget for NSA, full stop.
>As always it is helpful to remember as well that NSA's mission is to secure budget for NSA, full stop.
Sure, let's focus on an intelligence agency with budget constraints, Russia's GRU perhaps.
You claim that bug chains are "ludicrously cheap". Is cheap the same thing as abundant? If you had to guess, how many distinct zero-click exploit chains do does the GRU have for e.g. an iPhone in lockdown mode? Order of magnitude: do they have 1? 10? 100? 1000?
Zerodium pays up to 2M for "Full Chain with Persistence" for iOS: https://www.zerodium.com/program.html I don't think a low price relative to utility lets us conclude that such exploits are abundant. There's asymmetrical information in this market: buyers don't know the quality/novelty of what sellers have discovered, and sellers don't know how badly buyers need what they have to sell. It seems plausible to me that a savvy seller could negotiate a significantly higher price, similar to how tech workers are often able to negotiate significantly higher compensation -- especially if they were somehow able to prove that they weren't just replicating an exploit the broker already had in their inventory. I also suspect there is significant buying power on the buyer side which keeps acquisition prices low (hard to play buyers against each other, given low number of buyers who coordinate with each other).
In any case, I think this is the wrong question in a certain sense. The right question is about the relative cost of buying exploits vs developing in-house. I don't see why picking up the bug from email is hard or expensive. If the GRU is already running a program like XKEYSCORE, which seems likely, it could just be a matter of adding a few filtering rules for emails that go to select security@ email addresses. Have a GRU engineer monitor those emails, and see if any proof-of-concept work in the email can be quickly integrated into existing malware, in order to attack a target considered too low-value for the GRU's crown jewel exploits.
The real question is about the salary of that GRU engineer vs the cost of purchasing exploits. If the GRU engineer gets paid $100K, and a fresh exploit costs $500K, employing the GRU engineer to harvest a few temporary, expendable exploits a year looks quite favorable. I don't think the price/utility ratio of exploits from brokers affects the decision, since that price/utility ratio argument also works for exploits harvested+developed in-house.
Neither of us really knows what's going on in intelligence agencies, but my story seems about as plausible as yours. Given that simply using a Google Form for bug disclosures would be an easy and dramatic improvement on the status quo, I'm left with the sense that there is a lot of dysfunctional cargo-culting going on in the security world.
OpenBSD doesn't even have hyperthreading? Why does anyone use this OS? The Linux developers put in a lot of effort to make hyperthreading actually work for their kernel rather than ignoring it.
There have been cases where OpenBSD's hypothetical mitigations have worked out well for the project. I recall a relatively recent DNS cache poisoning attack that OpenBSD was novel in pre-emptively mitigating because something (I think it was the port?) was "needlessly" random.
If a mitigation has negligible performance impact, and doesn't introduce a new attack vector, I can't imagine why it would be seen as a bad thing.
That's from four years ago and does not address these technical issues. Are you going to pull it out every time OpenBSD is mentioned? I think people understand that you don't like their approach, etc., and the flaws you see, and that OpenBSD isn't designed for your interests.
Is there a current meta for OpenBSD exploit developers?
What's the right way to go about hardening the system if there's no meta to observe?
My very naive take would be something like: A successful exploit depends on jumping through a number of different hoops. Each of those hoops has an estimated success probability associated with it. We can multiply all the individual probabilities together to get an estimated probability of successful exploit -- assuming that hoop probabilities are independent, which seems reasonable? The most efficient way to harden against exploits is to try and shrink whichever hoop possesses the greatest partial derivative of overall exploit success probability with respect to developer time.
The meta doesn’t exist because nobody targets OpenBSD because it’s not used. People’s analysis of it is mostly just their educated guess as to how work for other platforms would carry over.
> The most efficient way to harden against exploits is to try and shrink whichever hoop possesses the greatest partial derivative of overall exploit success probability with respect to developer time.
Depending on your definition of efficient, adding more hoops should work exponentially better.
Suppose your hoop probabilities are 25% and that you have two hoops so that the probability of jumping through both is
25% * 25% = 6.25%.
You can reduce the size of one of the hoops in half, changing the probability to
25% * 25%/2 = 3.125%
You can also add a third hoop, in which case the probability is
25% * 25% * 25% = 1.5625%
1.5625% < 3.125%, so adding a third hoop is better than shrinking one of the two existing hoops. Of course, this argument makes important assumptions about the hoop probabilities.
The probabilities aren't independent. The person jumping through the first hoop is probably more able than average. Therefore, any additional hoop - if it doesn't require a completely orthogonal skill - is less selective.
I think it depends on what the "probability" is meant to indicate. You're correct if it's meant to indicate whether a particular attacker can get through a particular hoop. But probabilities could also refer to e.g. the chance that it's possible to get through a particular hoop, period. Or the fraction of some input space which corresponds to an exploitation.
Makes sense. Other key questions would be: complexity cost of added hoop (including, possibly, increased attack surface -- the sequence of hoops is just an abstraction that reality may not obey) and also creation difficulty (it could be that improving an existing hoop is significantly quicker than creating a new one).
https://twitter.com/halvarflake/status/1156815950873804800
With that in mind, it'd be handy to know which exploit techniques these steps break, and whether those steps are in the current "meta" game for exploit developers.
(The specific mitigation here: the kernel formerly locked system call invocation down to the libc.so area of program text in memory; libc.so is big, so now OpenBSD locks specific system calls down to their specified libc stubs; further, in static binaries, the same mechanism locks programs down to only those system calls used in the binary, which effectively disables all the system calls not explicitly invoked by the program text of a static binary).