Hacker News new | past | comments | ask | show | jobs | submit login
Apple curl security incident 12604 (haxx.se)
280 points by soraminazuki 11 months ago | hide | past | favorite | 62 comments



That behavior is completely stupid. I agree with Daniel on this, because if I choose to give my own CAs, there are two reasons: Either I don't have my CA in the OS bundle, or I want to validate specifically against a certain CA.

That means that this "feature" by Apple either adds more computation for nothing, or breaks the verification model that is expected.

Neither of these are an expected result.


Minor correction: it falls back to Apples CA store if the initial check fails. So wouldn’t add more computation for nothing.

However I do agree that it’s not the expected result and thus bad behaviour.

Given how Apple generally favour backwards incompatible changes and that this feature is added into curl, this does feel like there is more to this story than Apple is letting on. Perhaps it’s used for developer diagnostic tools or AppStore validation?


This is the kind of behaviour (corporate and software) that I would anticipate if some government has ordered them to secretly install it as a back door — a government-controlled item in the CA store (which isn't necessarily obvious, I think it just means at least one private key got copied), a requirement that all first-party software must use that store, and a requirement not to admit that this is a problem.


> Minor correction: it falls back to Apples CA store if the initial check fails. So wouldn’t add more computation for nothing.

Not GP but that is extra computation? It should just fail at that point, instead it's doing computation. According to the result, it's then either 'for nothing', or in order to give the incorrect result.


I disagree, but please read my comment through because if you take the first part of the comment alone then it might read as defending Apple, when I'm really not.

That's a failure scenario though. If your application is failing a CA check against an authority you've pinned then your infra and application is already broken in ways that the extra computation is meaningless. It is your certificates or application that then need correcting.

In terms of computational overhead (and only computational overhead!), where it matters is the happy path because That's where the overhead of additional computation would affect the performance for your operational infra and thus your users (and we are talking about this specific scenario and not some more generalised discussion about TLS). If you follow the happy path then Apples change, as serious as it is, doesn't affect the behaviour or your application.

I'm not defending Apple here. Whether intended as one or not, this change is a literal backdoor. And it needs to be framed as such. Arguing it as a computation overhead is, at best, missing the significance of this change in behaviour.

So the real problem isn't computational overhead, it's the question of what signing authorities are given permissions to write certificates for your pinned endpoints that wouldn't / shouldn't have had under curl's default code and expected behaviour. ie who now has a backdoor?


I did read your whole comment, and I didn't really take it as 'defending Apple', at least not in the sort of blind fanboyism way that implies.

If I understand correctly, you're assuming that the user is in control of both running the curl command, and the host it's pointed at? And thus the behaviour in this case is, you argue, arbitrary because you shouldn't let it get there anyway?

I was just thinking of it as the former, say I want to curl -X POST a comment to secretnews.ycombinator.com (not real, as far as I know!) and verify that someone hasn't MITM'd it so that I accidentally disclose my comment on news.ycombinator.com. I don't control either domain, I'm just running a curl command.


I’m saying calling this a problem because it might add a computational overhead is missing the severity of the problem.

This is a problem because it creates a backdoor.

I also disagree it creates a computation overhead because it’s only engaged outside of normal operation (ie when your application is already broken or if someone is intentionally exploiting that backdoor). but the computation argument is more of a distraction from the real problem.

So definitely not defending Apple. I’m saying it’s more serious than just a computation problem.


Hm, ok. I don't think they did miss the severity though, at least by my reading of:

> That means that this "feature" by Apple either adds more computation for nothing, or breaks the verification model that is expected.

they're saying one of two things happens: you get the wrong result (more severe than amount of computation) or you (because the cert it does have is not in Apple's store either) get the correct result, but wasted cycles doing something that you didn't want it to do and risked getting the wrong result.


You, not they. The “they” in this context is the author of curl. And Daniel is fully aware of the problems this change makes.

Whereas you’re the one barking on about computational overhead.


No, that's a quote from the top-level commenter (who wasn't me, as I said when I first commented), hence 'they', because they said that, not me.


My apologies. Full of confidence of at the moment so must have misread your comment.


Cold!! I’m full of cold, Not confidence.


Never attribute to stupidity that which could be explained by malice (especially if the malicious actor uses Hanlon's razor in defense).


Unfortunately, this kind of behavior, where the Apple policies always override whatever the "owner" of an Apple device wants to accomplish, is not surprising but it is what should always be expected from Apple.


This is also what keeps me from them. The devices seem really well-built and the OS polished, but with their lock-in to their own ways and own ecosystem (can't get access to your own hardware), it's just not compatible with being a hacker in the HN sense of the word, or at least not for that part of your digital life, and it's not like you'd want to buy it as a device on the side given those prices. I keep doubting about trying out Apple products, the latest one being their AR headset, but so far they've always seemed too hostile towards developers/tinkerers


Run open source curl. You now have the desired behavior.


Unfortunately, there’s always someone who is more than willing to read far too much into a low-level technical decision if it serves to, in their eyes, ‘prove’ their preconceived worldview.

A company as large as Apple making this particular decision in service of a wider vision to “own user devices, is attributing to Apple a degree of organisational and coordination prowess that is well beyond unheard of in any organisation even one tenth of Apple’s size.

But it’s Apple, right!?


It would be a lot easier to agree with you if Apple didn't go out of their way in software, hardware and firmware to block third-parties from getting a fair shake. This is the company that stops you from repairing your own device using software that you cannot control - it's not a lack of organizational and coordination prowess that makes these things happen. Nobody cakewalks their way into manufacturing things on the Chinese mainland.

Similarly, some intern at Apple did not authorize this change, write the patch, and then defend the decision when upstream questioned them about it. That requires coordination, which is frankly why this whole brouhaha is offensive. Apple could be working with upstream to close a critical vulnerability, but instead they're defending the merits of a faulty and dangerous fork. That is brazen and irresponsible behavior that requires just as much effort as working together with the community to solve the issue. Even companies a tenth Apple's size can get this right.


Apple is the owner, you are the user ;)


Presumably apple are setting this[0]? Emphasis mine.

  CURLSSLOPT_NATIVE_CA

  Tell libcurl to use the operating system's native CA store for certificate verification. If you set this option and *also set a CA certificate file* or directory then during verification those certificates are searched in addition to the native CA store. 
It seems as though if the --cacert and this option are combined, libcurl tries to honor both of them. Perhaps they should be mutually exclusive?


No, that's not the case. `curl` the binary knows how to call `libcurl` the library.


Right. I'm more suggesting that this combination of options should be handled differently inside of libcurl. E.g. either you want the native store for validation, or you don't. If you "also set a CA certificate file" as per the libcurl docs, presumably you want to use that cert only, and no others.

I'm not too familiar with how the library works internally, but I was imagining something like the above would be possible.


It isn't (lib)curl that's doing this, it's Apple's own version of LibreSSL that curl is built against on macOS.

There's nothing curl can do about it.


This is a backdoor.

I'm not saying it's intentional, or malicious. But as a matter of fact it is indeed a backdoor.

If you start adding keys to your users authentication scheme, then you've added a backdoor.


Seems like exactly the sort of thing that the NSA pressures large American tech companies to do.


Also the sort of issue which the large American tech companies conspicuously dismiss as unimportant.


As much as the default behaviour is questionable, I don't fully agree with the assessment: this is in fact a curl documentation problem.

As a multi-protocol lib, curl does not in fact implement all of them directly, instead relying on transitive dependencies which act as "backends" to outsource, in most cases, decoding the low-level bits. For some of the protocols, and for good reason, it incorporates support for multiple alternative libraries.

The downside of that approach is that it is hard, if not impossible, to ensure common behaviour across independent backends which may not cover all of the same features, provide incomplete APIs, or, such as in this case do not provide a way to patch some of its default behaviour. Libressl is not a bit-for-bit reimplementation of openssl, and are under no obligation to mimic its API fully (however convenient that would be for us all).

In such cases, and given the reluctance of upstream to fix it, curl is left with 2 possibilities: drop support for said library, or document the quirk. Given the downside of breaking user code of following the former, it should at least do the latter.

All this being said, I do agree with the main sentiment that this is a flawed approach from a security standpoint of libressl, and there's probably reason to open a CVE, but it should be on libressl.


The thing I'm not quite getting is how building curl from sources apparently gets around the issue, if the root cause is in the system ssl library?


The user who submitted the bug report was using Homebrew, which does not build curl with the system version of OpenSSL: https://github.com/Homebrew/homebrew-core/blob/547d6bb9b289c...


Reminds me of the SQLite F_BARRIERFSYNC debacle.

They just don't care.

https://bonsaidb.io/blog/acid-on-apple/


I remember a while back when they broke one of the ancient Unix tools they actually ship by hacking in some option badly. Their attitude to maintainership is like two levels below a Debian guy breaking OpenSSL.


If Daniel says your curl is broken, you fix it, Apple. It’s that simple.


Daniel can be wrong. He can even be wrong about curl.

He's probably correct here, as far as I can determine in 2 minutes, but "he's correct because it's Daniel" is the worst possible argument.

This reminds me of this old exchange about the history of C (specifically, Eric S. Raymond's "contribution" to it): https://scienceblogs.com/deltoid/2011/10/14/dennis-ritchie-h...


I think the counter-argument is that "curl" is a project maintained by Daniel. If you are going to ship a product that doesn't behave in the same way you should call it something else. So Daniel is right about what "curl" should do, but Apple can be right about what "applecurl" which is very similar to curl should do.


I think it should warrant more dialogue than that first line support shut down though, even if the Apple team's gut reaction is to disagree.

Not 'Daniel' specifically, but in general when you're packaging a tool and its author/maintainer(s) contact you about it.


He’s been maintaining an essencial part of the computing infrastructure for 25 years and doing a great job at it.

A fair coin can land on heads 20 times in a row, but the odds are not great.


I think parent doesn't mean Daniel is never wrong even without proof but if one side claims a bug is a bug and the other says it's not than you can pretty sure it's a bug if Daniel considers it as one.

It's about the evaluation of a finding.


Thanks for the warning! FWIW, I'm using MacPorts to override many of the tools that come bundled with macOS (curl being one of them), because they tend to be out of date or otherwise broken. I've long since lost trust in Apple's bundled software.


I wonder if Apple relies on that behavior for something important...


Not sure if you're being snarky, but obviously this is very very important.

It'd be entirely reasonable for someone to create a script that is meant to only use the internal private CA. When you run this command then you KNOW that you are ONLY talking to an internal corporate resource, because this is an internal corporate CA.

And then apple adds a backdoor the size of domain validation, to this.

Hell, it'd be entirely valid to use a dummy name, and just rely on the fact that it's signed by the corp CA, to establish that you are connecting to a corporate server. Now Apple's added a security vulnerability to that, by breaking this extremely reasonable assumption.

Not cool.


Making corporate/governmental TLS introspection easier.


It must either be something extremely stupid or nefarious.


Or laziness, eg hacky, which is I guess in category stupid.


So much about Apple caring for the security of its users.


> Obviously I think differently.

:)


A default behaviour is not the same thing as a fallback behaviour. Seems that Apple's security team has a reading comprehension problem.


I guess it's a general MIT license problem. Somebody, be it Google, Apple or whoever else, can deliberately break your program and distribute it under the original name. And although you kinda own the name you really don't.


It's not. You can do the same with GPL-licensed code. You can use trademark law in both cases to prevent people from distributing modified versions, like Firefox did for years with Debian

https://en.wikipedia.org/wiki/Debian–Mozilla_trademark_dispu...


Wouldn't the same problem happen with most other (if not all?) FOSS licenses?


Would it be theoretically possible to counter this with a trademark? And then say (in legalese) "if you're just packaging it then I grant you permission to use the trademark, but if you fork it (or otherwise substantially modify it) then permission is revoked"



It’s apple. There standard operational procedure seems to be it’s not us it’s you whenever you question their stupid policies.


[flagged]


> If you stay inside, and that's a big if, using Apple is bliss.

Not so much anymore really. There's lots of bugs, even if you stay inside the Apple way.


Sadly true, though there always has been bugs in the Apple ecosystem, much more than everybody wants to remember.

I remember my dad telling me when I was little “Apple is not what it used to be; now there are so many bugs. I miss the good old times…”

Personally I think the main issue with Apple is the lack of transparency wrt what actually happens. Take AirDrop for instance. It’s magical when it works, but truly opaque and frustrating when it doesn’t…

The other issue is the follow-up for reported bugs which is lacking, though I guess they have a lot of reports and following up on everyone of them must be tricky.


> It’s magical when it works, but truly opaque and frustrating when it doesn’t…

That is very much my experience as well.

I‘m pretty savy in macOS, Windows and Linux.

While macOS is certainly the most beautiful of these systems, it’s also an absolute pain to troubleshoot sometimes.


[flagged]


Curl has the ability to use the OS certificate store. There is also the option (at invocation) to not use any other certificate stores than the one provided by the user (at invocation). The version which is shipped by Apple does ignore this which introduces a backdoor.


That behaviour is also a curl configuration option i.e. with-ca-fallback.

Curl definitely should be updating man pages if it is falling back to OpenSSL CAs when --cacert is specified.

Homebrew Curl on Mac also sets this flag:

https://github.com/Homebrew/homebrew-core/blob/9cccce7a6dff7...


curl (proper) doesn't do that, nor is it supposed to, that's the point


This behaviour is part of curl (proper) and is set via compile time flags.

And at least according to curl's code is set by default when it is built against OpenSSL:

https://github.com/curl/curl/blob/1ccf1cd9936dfa382fe1f061b6...


It's not set by default. (and even when it is set, does it modify the behaviour of the --cacert option?)


You mean Apple, who modified the code to work in this broken way.


Oh yes, you should definitely write to the author of the article above correcting his understanding of curl and suggesting he raise that issue.

Don't forget to use your Apple corporate email address.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: