Hacker News new | past | comments | ask | show | jobs | submit login

I had this conversation with a person on reddit just the other day. After I pointed that there is no difference between piping a script directly into bash and downloading a package and installing it, they told me that they can audit a script/package they download and that they have personally audited every bit of code on their linux box. The kernel, gnome and metasploit.

I got to thinking what it would take to do such a task, and mainly what I came up with was a whole damned lot of time. Assuming a person can read, understand and remember 5000 lines of code per day (which honestly I think is way more than is realistic), it'd only take 8.5 years to audit the linux kernel. Add in all the other stuff and I figure something on the order of two decades. And in the end you'd be running two decade old software or you'd have to start over.

At the end of the day, security comes from the personal and corporate economics of reputation, profit, and prison avoidance. Do your best to get your stuff from people who you judge to be trustworthy and rely on their own self interest to not be malicious and to do their best to protect their repositories. And rely on others to be good citizens and report the bad shit that happens to them so that things can be cleaned up.

Now I'm not saying to throw out security best practices, but people should be aware that the quality of their systems are built on trust in human nature and self interest.




What if the download gets interrupted due to network error and bash runs something else than what was intended. E.g. the script has:

  rm -rf /tmp/something
But when the pipe gets interrupted it executes:

  rm -rf /


The body of the shell script should be wrapped in a function which is invoked at the end. If you do not trust them to get this incredibly basic thing right, how can you possibly trust any of the other code they have written?


sandstorm.io said: «If the connection dies mid-line, bash will execute the incomplete line – however, this can be mitigated by wrapping the whole script in a bash function executed on the last line, as we do.» (Source: https://sandstorm.io/news/2015-09-24-is-curl-bash-insecure-p...)


The same way thousands of people trust Steam. You would expect a company of Valve's size to get their Linux uninstaller right, but people found out the hard way that it was removing their entire home directories.

People make mistakes. Mistakes contained to programs is one thing. Mistakes in shell scripts tend to have bigger consequences.


Soo, maybe a silly question... but are downloads from the internet always "linear"? Meaning they go uninterrupted from start to end and if it drops it will stop in the middle of the downloaded file? Doesn't it get "pieces" and then place them together?


It depends on your download program, but that's irrelevant because of the pipe. It has to be linearized before going through the pipe.


Packets can arrive out of order, yeah. TCP includes ordering information, so lost packets can be retransmitted and in the end everything'll be in proper order.

As for UDP, I'd tell ya a UDP joke, but get it you might not.


It does get pieces, which are sequential. At least, curl does, if you don't tell it otherwise.


The https://github.com/silentbicycle/curlbash utility can check the hash of the script and refuse to run it if there's a mismatch.


There are better strategies than blindly relying on third party reputation. A primary approach is package management with hash databases, to ensure you are receiving the same code as others. Another is the use of multiple implementations whose results are compared, recently discussed @ https://news.ycombinator.com/item?id=12666923


You're still relying on the source to not be underhanded and malicious. Getting the same, reproducible build as everyone else only means you've got a lot of company. It doesn't mean there isn't malicious code or intentional "bugs."


You are right, though with hashing you've got effective post-release protection against a compromised source injecting malicious data at the very least, which these days is probably more likely than an outright malicious source for most major packages. So it's not without value. The second approach, more of an architectural workaround than a true solution, can still be used to insulate from major issues.


You can do that with curl + bash too. RVM does this: https://rvm.io/rvm/security


> there is no difference between piping a script directly into bash and downloading a package and installing it

Well then I guess you haven't seen this, in which the server detects whether the script is being downloaded to disk or piped to a shell, and adjusts the payload accordingly:

https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...

i.e. when you download to disk and look at it, it looks benign, but when you pipe it to a shell, you get owned.

Just say no to curl|sh, kids.


People who pipe it to a shell aren't going to look at it anyway. That's kind of the point. If they were, they'd download it first. There is literally no security gained by downloading it and running it, instead of just `curl|sh`. IF it's malicious, you just installed it either way. The only thing downloading does in this scenario is lets you look at the script later on when you realize it might have been malicious, but that's even assuming you kept it (I would wager that almost everybody would throw away the script after installing it if they weren't planning on reading it before installation), and assuming you know enough to understand its contents (not everybody knows bash scripting, and you can write some surprisingly complex scripts).


No, the point is that installing software by either downloading and executing shell scripts or piping shell scripts directly to the shell is a wildly bad idea.

And the ancillary point is that piping directly to the shell is worse, because it allows an attacker to exploit any false sense of security a user might have from having looked at the downloaded version of the script (whether that's that user specifically or someone else who did so and vouches for it).

The correct thing to do is distribute via cryptographically signed archives or packages, or via signed git tags.

Again, just say no to curl|sh, and also just say no to downloading and executing unverified shell scripts.

Really, what are you arguing for here? Security is a process that involves layers, and you seem to be advocating for tossing all the layers aside because layer A doesn't protect against attack method B.


It's not a wildly bad idea. You are seriously out of touch with how most people use computers if you think that. Very few people actually read every script they execute, and if it's a binary exe, they can't even do that. Whether or not you choose to execute software you downloaded depends entirely on your trust in the vendor and in the delivery process. If you trust the vendor to provide you with something you're willing to execute, then it really doesn't matter how you go about executing it either. Downloading it first and then executing it is no safer than piping it to bash. All it does is give you the option of reading it first, but as I said above, almost nobody actually does that.

> The correct thing to do is distribute via cryptographically signed archives or packages, or via signed git tags.

And now you have the problem of trusting the key instead of trusting the delivery mechanism. This approach works well for package managers because the user's trust in the package manager implicitly trusts the keys the package manager uses to verify all the software it downloads, but it doesn't work all that well for software distributed directly to end users, because very few people even know how to verify the signature, let alone figure out whether or not they should trust the key it was signed with. After all, the fact that it's signed doesn't mean anything, what's important is whether it's signed with a key that is trusted as belonging to the vendor (and is assumed to not have been compromised).

> Security is a process that involves layers, and you seem to be advocating for tossing all the layers aside because layer A doesn't protect against attack method B.

"Read the script before executing it" is a really really crappy layer. That assumes that you understand shell scripting, that the script is simple enough to actually be reasonable to skim through it and understand what it's doing, and that the hypothetical attacker hasn't figured out how to disguise their modifications to make it not immediately obvious to someone skimming the code. As previously stated, almost nobody actually does this. And once you've determined that, no, you're not going to read through every line of the shell script that you're using to install the software, then `curl | sh` is perfectly fine as long as you trust the delivery mechanism (e.g. https). At this point the only thing you've lost is the ability to verify code signatures, but people don't generally sign their shel scripts anyway, and also as previously stated, even if they did not very many people understand how to actually validate it and how to verify that they have a trusted key. GPG is really not user-friendly at all.

I guess what it really comes down to is, use a delivery mechanism that handles all this for you if you can (such as a package manager). If you can't, then the only real question is "do I trust this vendor and delivery mechanism enough to run something I just downloaded on my machine?"


You're just repeating the same old fallacy that comes around every time this issue is discussed: "Most people don't or won't or can't verify the files they download, so there's no reason to try."

I'm sure bad guys and state-level actor-types love it when people like you encourage people to continue promoting Worst Practices for security. Anyone who can MitM can trivially exploit this to take control of any machine that installs software this way, and they can do so surreptitiously by exploiting the technique that sends different content depending on whether it's piped to a shell or saved to disk. And guess what: when it's piped to shell, the evidence is gone as soon as the shell process exits. It's like the perfect crime, and thanks to people like you who expend significant effort convincing people to keep doing it, it continues to be a problem.

Oh, and you apparently haven't even considered what happens when a connection is interrupted and the shell receives a corrupted or incomplete script. This alone is reason enough to never do it.

The correct thing to do is to discourage Worst Practices and encourage Best Practices. The correct thing to is to never tell people to pipe to shell. The correct thing to do is to provide instructions on how to verify the integrity of downloaded software. To do anything else is grossly negligent and irresponsible.

> the only real question is "do I trust this vendor and delivery mechanism enough to run something I just downloaded on my machine?"

It has been demonstrated that the delivery mechanism in question is completely insecure. Even TLS does not ensure that a state-level actor who can forge certs is not MitM'ing a connection. You think NSA can't do this? They probably can. Can they break a 4k GPG key? Unlikely.

So the real question here is why you're trying so hard to encourage people to continue with this grossly irresponsible practice.


You're arguing that `curl | sh` is insecure as compared to something that literally nobody does, which is to read every single line of every script they ever execute. Once you discard the ridiculous notion that everybody should be reading every line of the script before running it, then there's nothing wrong with `curl | sh`. Yeah, you can't go back and read the script later, but if you wanted to do that, then just don't use `curl | sh`! If you're not planning on going back to read the script later, you wouldn't be saving it anyway after you ran it, so I genuinely don't understand how this argument is supposed to be convincing.

> Oh, and you apparently haven't even considered what happens when a connection is interrupted and the shell receives a corrupted or incomplete script.

I addressed this in my very first comment. You might want to try actually reading it before accusing me of not understanding an issue that is honestly really trivial to fix (wrap the whole script in a shell function and then execute the function; early termination means syntax error without anything being executed).

> The correct thing to do is to discourage Worst Practices and encourage Best Practices. The correct thing to is to never tell people to pipe to shell. The correct thing to do is to provide instructions on how to verify the integrity of downloaded software. To do anything else is grossly negligent and irresponsible.

Refusing to provide a convenient way to install your software doesn't make anyone safer, it just means you have fewer users. Anybody who is even remotely able to read shell scripts or to verify software integrity already knows how to read an install instruction like `curl | sh` and modify that to download the script first. And nobody who doesn't already know how to evaluate something like `curl | sh` and decide if they want to do that or download/verify the script first is going to bother using your product if you make the install instructions overly complicated.

> It has been demonstrated that the delivery mechanism in question is completely insecure. Even TLS does not ensure that a state-level actor who can forge certs is not MitM'ing a connection. You think NSA can't do this? They probably can. Can they break a 4k GPG key? Unlikely.

As stated in the original comment, even considering a state-level actor who can forge legitimate certificates is way out of scope for this discussion. Anybody who is at risk of being targeted by a state-level actor already has to take many many precautions that normal users don't take, and your website telling them to use `curl | sh` to install something is not going to be an attack vector, because either they understand how to download and verify the script first, or they've already been compromised by any number of easier ways than hoping the target decides to install the particular piece of software that the state-level actor has compromised.

> So the real question here is why you're trying so hard to encourage people to continue with this grossly irresponsible practice.

No, the real question is why are you behaving as though users are simultaneously capable of reading every line of every shell script, doing GPG verification of downloaded software, and figuring out how to even make sure the GPG keys are trustworthy, while at the same time so ignorant that they can't figure out how to go from `curl | sh` to "download, verify, and then run" without step-by-step instructions.

Avoiding `curl | sh` does not make anybody safer. All it does is make it harder for users to install your software, while not changing a single thing for users who are paranoid enough to want to verify the software before running it (since they can do that anyway). The only 2 things that you need to do to make `curl | sh` reasonable are always use https, and make your script resilient against unexpected EOF.

Edit: Another thing that hasn't even been considered this whole time, is if the threat model here is the website or delivery mechanism has been compromised, then the attacker can change the install instructions too. So even if you don't offer `curl | sh`, the attacker could modify your install instructions to use `curl | sh` and the victim wouldn't know that's not the "real" instructions. The alternative threat model is the vendor itself is untrustworthy (e.g. they're serving up a malicious script without having been compromised), but there's nothing you can do to guard against that because if you don't trust a vendor to not intentionally give you a malicious installer, then you shouldn't be running their software to begin with.


People paranoid enough to download it and read it are more likely to run what they just read. Why would you download it again?

curl >x cat x . x

However, the point is that if you can't trust the publisher it doesn't make any difference what the installation procedure is.


What? I feel like you're not thinking about this very carefully. Someone might look at the downloaded script and verify that it is harmless. Then they might run it. Then later they might go to another system they want to install it on, and they might think, "Well, I looked at it a few minutes ago, and it was fine, so now I'll just pipe it to the shell like the web site says to do." Or they might recommend the software to someone else, and they might say, "Hey by the way, I verified that the install script is safe, so you can pipe it straight to a shell like the site says."

> However, the point is that if you can't trust the publisher it doesn't make any difference what the installation procedure is.

That's not correct. A MitM attack is a risk even if the publisher is trustworthy.

This is not complicated--in theory, at least: only distribute cryptographically signed software. Anything else leaves you vulnerable to many attack vectors. And anyone who instructs users to pipe to a shell is irresponsible and encouraging Worst Practices.


I feel like you missed the point of what I wrote. Yes, there are things people could do where if the site owner is malicious they could get screwed. That is generally what happens when you install software from untrustworthy sources.

> > However, the point is that if you can't trust the publisher it doesn't make any difference what the installation procedure is.

> That's not correct. A MitM attack is a risk even if the publisher is trustworthy.

You're contradicting something that I didn't write.


Sorry for the formatting:

  curl >x
  less x
  . x


A downloaded installer can be signed, and you can verify signatures or at least checksums out-of-band.


That just asserts that it's the same series of bytes that was originally provided. You still have to trust the provider.


Right, we know that security is a gradient, do you?

There is a quantitative difference between covering your laptop under a coat and locking the car doors and leaving it on bench when you take a walk around park.

Just as there is difference between piping the internet to root shell and installing a signed package that has whole ecosystem and many users validating and looking at it.


In fairness, ssl from a trusted host is pretty similar.


You're trusting the provider of the program, not the provider of the server. Curl/bash from github? You're trusting Github to provide the correct content; there have been times before where people have been able to spoof other users. Signatures on the package (or the code) would prevent an issue like this.


You always have to trust the provider. If you don't authenticate the message you also have to trust every carrier.


At least you avoid javascript pastejacking in this manner.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: