Now Microsoft just sounds like pre-Brexit Britain. Why reflect on your own shortcomings when you can blame the EU instead :)
I suggest Microsoft follows Britain's example and leaves. The main difference is that we Europeans actually miss the Brits, whereas nobody would miss Microsoft and its shoddy products and business practices.
On a more serious note, I fully understand that the Digital Markets Act is causing Microsoft headaches. But I think this headache is well deserved. Big Tech has been building moats where they should have built bridges, and now our computing landscape resembles medieval Germany where everything was at the mercy of a few feudal lords. It is time to drive out those lords and reshape software in a way that empowers, not enslaves.
Apple did, Microsoft is working on eBPF for Windows[1] but I doubt they'll sunset their kernel modules support. At the very least, it means there are safer ways to load third-party code in the kernel without allowing them to crash your entire system by mistake. Even if kernel modules are still supported, a compliance framework may introduce a "No kernel module" requirement, just like they require a CrowdStrike-like software to be installed.
However, doing so is no easy feat. The first version of eBPF was released over 10 years ago.
Apple did not have to sign a settlement with the EU, which Microsoft did in 2009.
The terms of the settlement state:
"Microsoft shall make available to interested undertakings Interoperability Information that enables non-Microsoft server Software Products to interoperate with Windows Server Operating System on an equal footing with other Microsoft Server Software Products." [0]
"Microsoft shall ensure on an ongoing basis and in a Timely Manner that the APIs in the Windows Client PC Operating System and the Windows Server Operating System that are called on by Microsoft Security Software Products are documented and available for use by third-party security software products that run on the Windows Client PC Operating System and/or the Windows Server Operating System. These APIs will be documented on the Microsoft Developer Network, unless open publication would create security risks. In such circumstances, Microsoft will provide third-party security vendors with access to such APIs pursuant to a royalty-free license and on fair, reasonable and non-discriminatory terms." [0]
This means that by offering Microsoft Defender for Endpoint, Microsoft needs to give similar access to the underlying kernel to competing vendors like CRWD and S1.
> At the very least, it means there are safer ways to load third-party code in the kernel without allowing them to crash your entire system by mistake
An eBPF or a similar technology wouldn't necessarily enhance stability when probing kernel-land from user-land, as any interaction with a kernel can cause kernel panics.
Plenty of endpoint vendors have had issues with PTrace (MacOS), kprobes (Linux), eBPF (Linux, K8s), etc.
The reality is that $))&#( happens, and a lot of comments on HN about this bug are really dumb.
> Plenty of endpoint vendors have had issues with PTrace (MacOS), kprobes (Linux), eBPF (Linux, K8s), etc.
Nothing is perfect, latest generation of Intel CPUs are unstable, bugs are part of all kernels, and so on and so forth. But eBPF does allow for less crashes than a kernel module and these crashes are usually sandboxing error rather than an error made by the third-party provider.
As a matter of fact, the eBPF Linux version of Crowdstrike did create kernel panics on some distros.
> This means that by offering Microsoft Defender for Endpoint, Microsoft needs to give similar access to the underlying kernel to competing vendors like CRWD and S1.
Absolutely, and that's a good thing. This doesn't seem to prevent them from moving Defender to another set of security APIs or from creating another set of security APIs for anyone to hook on.
Nothing in that wording says that Microsoft can't restrict access to kernel modules, as long as they don't give their own product an exception to that restriction. They had the choice to give Windows APIs that allow to implement such software in a safer way and use them for their own product, but choose not to.
"Microsoft shall ensure on an ongoing basis and in a Timely Manner that the APIs in the Windows Client PC Operating System and the Windows Server Operating System that are called on by Microsoft Security Software Products are documented and available for use by third-party security software products"
Everything is an API. This means they cannot restrict access to the underlying kernel.
> They had the choice to give Windows APIs that allow to implement such software in a safer way
An indirect method would still inevitably have a potential issue. If a third-party vendor made a mistake, it's the third-party vendor's fault - in this case CRWD.
You can create an appropriate API to run the code outside the kernel. Nobody is forcing MS to run everyone's the security code in the kernel, MS choose to (because it's easier). The law is about fairness and equal access for all. Apple and Linux have shown alternatives.
Sure, it's safer if only a single vendor is allowed to do this but it's also unfair.
> that are called on by Microsoft Security Software Products
Nothing says that Microsofts Security Software needs to be implemented in a way that runs in the kernel, which is inherently more vulnerable. Other platforms have APIs for security software to hook into kernel actions without directly running in kernel space (Windows does too, but to my understanding more limited to logging, not intercepting activity, and hence limited in usefulness).
Hence the claim that it was impossible for Microsoft to prevent this is false. That doesn't mean they take all the blame for someone elses mistake, but it still means that they made a choice leading to it: Deciding that security software, whoever made it, running in kernel modules was the way to do it.
Why would Microsoft even bother making this comment? Is the outage in some part their fault? I was under the impression it had everything to do with the botched croudstrike update, and nothing to do with Windows itself. This could have just as well happened with some widely deployed antivirus running in the Linux kernel.
The headline is clickbait.
Microsoft is saying why they couldn’t secure the kernel against such an attack, and are right in saying that the EU prevents them from closing it off to third parties.
They are not saying the EU is the root cause of the failure, just that they cannot close the hole currently due to the EU.
What they leave out is that they could choose to integrate Defender into the OS for free, thereby removing it as a product to compete against. They could also move Defender to not require kernel hooks either. Neither are options they want to consider currently.
Integrating Defender sounds like it would create an antitrust issue? If I remember correctly, MS was in the past taken to court and forced to sell some product or other separately, when they previously provided it for free.
No comment about being able to move Defender to not require kernel hooks (I don't know).
It’s unclear imho if it would be an antitrust behaviour. Many regions have security exceptions as long as it’s a core feature. It also wouldn’t prevent competitors working at a higher level either.
> Why would Microsoft even bother making this comment? Is the outage in some part their fault?
Two reasons:
1 - Few people understand anything about how their computer (/car/stove/phone/medicine/...) works -- they spend their time on other things.
Without any model of how their device works its easy to misassign responsibility (see how many people think that Safari is Google or vice versa). So it's in MS's interest to try to get the message out. Of course people do this when they are at fault as well.
2 - EU is in a wave of beating up* on certain large companies. This can also be an opportunistic way to push back.
* I am not implying whether I think the EU is correct or not.
MS is damned if they do damned if they don’t. You can already see it in the comments. “They had 15 years to fix this”, “this na an excuse”, “they are already on the attack” etc.
If MS had blocked these type of things people would be in here complaining about antitrust and MS is evil.
HN and in general the software community has a hate-boner for Microsoft, it is almost tradition at this poin.
While the hate is valid in many cases, I've observed that the cribbing about it has also been unwarranted or unjustified a lot of the time (also no other corp is held to the same standard) - and this is a prime example.
MS cannot legally restrict third party kernel. Apple can, bc they didn't get struck down like MS did.
MS has an option to not bundle Defender with their OS, which would let them lock the kernel to avoid the anti-trust restrictions, but that would be an insane decision to make.
Every time something like this comes up - people are unwilling and incapable of comprehending regulation has consequences. This is one of the consequences...
Our EU friends really enjoy having all the regs on everything... but then demand to be treated as-if the regs don't exist. It's amazing to see...
Microsoft is stuck because anything they'd do would reduce the "freedom" of end users.
I work in big tech, and unfortunately we frequently need to have conversations about the smallest features because we have evidence about us giving users an inch and they taking a mile.
The EU prays for a MSexit from the EU. The efficiency gains by that would be enormous. If we could just get also a SAPexit, Europe would become unbeatable.
From what I can tell, Microsoft signed DigiCert's certificate, and DigiCert signed CrowdStrike's certificate, and CrowdStrike signed the driver file.
Windows kernel signing does not work like Apple's Big Brother approach, it uses a set of certificate authorities.
Microsoft does have a program for verifying drivers: WHQL, which you may recognise from the slower driver that Windows Update installs for your GPU before you download the faster one that didn't pass Microsoft's verification from the manufacturer's website. CrowdStrike doesn't seem to be WHQL-certified.
A ridiculous excuse. They could have provided an non-EU Windows version if that's the case or better yet, create a robust solution as a software vendor to the said security companies.
People interested in running CrowdStrike would be forced to use the EU version anyway, and people uninterested wouldn't be affected by the Crowdstrike bug whether they used the EU version or not, so I don't see exactly how having two versions of Windows would help here.
What a disingenuous argument by Microsoft - I really hope nobody buys it. Lots of the most mission critical software runs on Linux. They don't have these security issues because they were open from the start.
My understanding is that Linux is just as vulnerable to this issue as Windows but isn't used as often for desktop software. In fact there two security incidents just this year involving Crowdstrike on Debian and Rocky Linux.
I see, reading about this issue it looks like eBPF can be used in Linux to sandbox kernel applications, preventing such applications from crashing the kernel as a whole.
Meh - Microsoft can point to the fact that they saved the day by preventing every Linux device from getting backdoored with the `xz-utils` debacle. The fact that it was caught was an exceedingly lucky break, and had nothing to do with quality engineering, or even that the code was open source.
I think the fact that it was caught is a testament to the power of open source and the quality of engineering. It speaks volumes about the promise of OSS.
> I think the fact that it was caught is a testament to the power of open source and the quality of engineering.
Ha, no. It was caught because an engineer noticed his SSH was taking slightly longer than normal, a few hundred milliseconds of difference. There was nothing in the code to suggest an anomaly; so he began fully reverse-engineering the binary as though it was proprietary software. The open-source communities meanwhile had approved and were even distributing that code on testing branches. And to top it all off, that engineer worked for Microsoft; so it's pretty ironic to complain about Microsoft's behavior with Windows, considering they just saved Linux from catastrophe.
wait, did the open source community have access to Microsoft's code and then fail to find the Cloudstrike vulnerability even though they had the same amount of time with that code that MS had with theirs?
I mean open source idea is that even MS engineer can find their problems.
And then MS engineer found their problems.
So yeah, does seem like a win for open source ideas which is not even something I particular care about...
> wait, did the open source community have access to Microsoft's code and then fail to find the Cloudstrike vulnerability even though they had the same amount of time with that code that MS had with theirs?
Cloudstrike was not in Microsoft's code; so yell at Cloudstrike.
> And then MS engineer found their problems.
The Microsoft engineer found the malicious code from the binary first - the same way he would have investigated proprietary software. The fact that it was open source didn't help discover the vulnerability in any way. The open source nature only helped with explaining how it got in there afterwards.
> So yeah, does seem like a win for open source ideas which is not even something I particular care about...
According to many security researchers, the `xz-utils` thing was deeply underrated and shows how, let's just say, not necessarily more secure open source software is. The fact that every Linux computer was nearly backdoored, globally, and it was found by accident, by a Microsoft engineer nonetheless, without any use of the code to find the bug, after that code was approved by both Debian and Red Hat, looks terrible to the open source community.
I take that back. It doesn't just look bad - it is bad. The ideology and security assumptions are in question, bad. If Microsoft's engineer had not found that bug, Linux and open-source as a whole could have sustained a mortal wound and a crushing blow to the theory of open-source being more secure.
Linux outages and upgrade fuckups happen all the time. The difference is that Linux isn't on a unified upgrade cycle so issues come as a trickle rather than a dulge, so the problems are usually localised to a corporation and don't make the news.
Also Linux fanboys will usually blame the system admin for not configuring things properly if things break: "it's not the operating system, it's <something stolen from OpenBSD>".
End of the day Linux is only popular because of the inertia UNIX had on mini-computers/servers. For standard end users GNU Linux is lightyears behind Windows and macOS in terms of usability and stability.
I suggest Microsoft follows Britain's example and leaves. The main difference is that we Europeans actually miss the Brits, whereas nobody would miss Microsoft and its shoddy products and business practices.
On a more serious note, I fully understand that the Digital Markets Act is causing Microsoft headaches. But I think this headache is well deserved. Big Tech has been building moats where they should have built bridges, and now our computing landscape resembles medieval Germany where everything was at the mercy of a few feudal lords. It is time to drive out those lords and reshape software in a way that empowers, not enslaves.