
Detecting the use of “curl – bash” server side - ingve
https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-bash-server-side/
======
ithkuil
> Installing software by piping from curl to bash is obviously a bad idea and
> a knowledgable user will most likely check the content first

It's funny how so many obvious things are all but obvious when you think a
little bit more about it. Interesting read on the subject:
[https://sandstorm.io/news/2015-09-24-is-curl-bash-
insecure-p...](https://sandstorm.io/news/2015-09-24-is-curl-bash-insecure-pgp-
verified-install)

(I don't want to enter the `curl | bash` good or bad rabbit hole; just that
the topic cannot be just dismissed as "obvious")

~~~
kenny_r
For me personally, the security aspect isn't even the reason I dislike curl |
bash installers. There's no standard behaviour for curl|sh, and it re-
introduces many problems that were solved by package managers decades ago.

Each time I'm about to run `curl
[https://install.newhotness.io|sudo](https://install.newhotness.io|sudo) sh`
I'm left with the following questions:

* Will it work? I'm on Arch Linux. This installer was probably tested on OSX and Ubuntu and deemed "portable". I've seen curl|sudo sh installers trying to apt-get installing dependencies.

* Where does it place files? I've got a ~/.meteor folder that's 784MB. Why would you install software libraries and binaries to $HOME? I now have to tell my backup tool to ignore it when backing up my home partition, great. The FHS was established for a reason, people.

* Correlary to the previous question: How do I uninstall it? Maybe all files were installed to a hidden folder in ~/, maybe there's some stuff in /opt, or /usr/local, or who knows where else. If the installer doesn't implement uninstall functionality itself, I now have to go hunt for the stuff it placed in my filesystem.

I understand the need to be able to distribute your software without having to
implement native packaging for OSX/Debian/Ubuntu/Red Hat/Fedora/Arch/...

Docker solved this problem. If you're feeling lazy when it comes to
distributing your app, just ship a Docker container and be done with it.

~~~
TeMPOraL
> _Correlary to the previous question: How do I uninstall it? Maybe all files
> were installed to a hidden folder in ~ /, maybe there's some stuff in /opt,
> or /usr/local, or who knows where else._

How does one uninstall stuff on Linux anyway? My Linux boxes are always like
thrash cans - I add and add new stuff until it's time to upgrade either the
system or the machine, so it gets wiped and replaced. Never figured out how to
delete stuff in a way that will not leave a ton of things behind...

~~~
serf
>How does one uninstall stuff on Linux anyway?

with a package manager suited for your system, or with the knowledge of what
files the software in question touches.

~~~
RijilV
It's a best effort though, you're relying on the package maintainer to have
tracked all the installed files and clean up anything they might have
modified, which hopefully you or something else hasn't modified after the
package was installed.

The packages authored by distributions tend to be pretty good, but I've seen
RPMs with nothing more than tar zxf in them and no tracking of the files
installed.

The major package managers expect the package to manage the artifacts
installed, this works if everyone plays nice and does their job...

~~~
alphapapa
You're right that this requires well-maintained packages, and this is one of
the strengths of Debian: its policies that require well-maintained packages.
If you install a package that's part of Debian, you can count on it being
well-maintained, as opposed to one put out by a random web site or developer,
which may install but may not integrate well or clean up after itself.

------
Velox
The unfortunate problem with this is that while piping directly into bash can
be exploited, it remains as one of the easiest ways to install programs.

Taking RVM for example. Their instructions are to run this: `curl -sSL
[https://get.rvm.io](https://get.rvm.io) | bash -s stable`. The script that is
executed is 887 lines long. The installation is "complex", requiring a lot of
different stages. Now, the solution to this is "Use a package manager". Sure,
that works in a lot of cases. However, when you have something like RVM which
is used across several major operating systems, and hundreds of different
flavours, each with their own quirks and package managers it suddenly gets
difficult to manage each of these.

The problem we face is, how can we make it easy to install something, while
still being safe and maintainable?

Breaking this down further, there are 2 issues to solve. The first is "How do
we ensure what we download is what the maintainer says that we should
download?". I.e. How do we make sure there are no malicious injections. That
one is simple. Use SSL.

The second issue is, "I want to install this thing but I don't know if I can
trust the installer". Are you crazy!? This isn't an issue. If you don't trust
the installer, you sure as hell can't trust the product. If you don't trust
either of them, then you automatically don't trust the other and shouldn't be
installing it.

The result is that, yes, people can maliciously serve up code when you pipe
the output of curl through bash without you realising. However, this is no
different than blindly trusting and installing a script.

~~~
geocar
> If you don't trust the installer, you sure as hell can't trust the product

I can't think of ten pieces of software with excellent installers.

Software distributors generally pay very little attention to the installer.
That is because installers are written by people who want to try and make it
easy to install something, and don't really care about anything else. If they
can get you to install something, helping you remove it _isn 't their
problem_.

If they can get you to install something, protecting you from really unlikely
things like someone hacking their CDN and delivering malware _is a high
quality problem_ : Either they have enough users so that they will be
forgiven, or they won't have enough users and the project is abandoned anyway.

I don't trust installers.

I don't trust installers to document what they're doing, or tell me where
files go, because they don't.

I don't trust installers to deliver a secure transparent experience, because
they don't.

I don't trust installers to consider conflicts, like _what else do I have
installed_ because they don't.

I don't trust installers to create security boundaries, protecting me and my
files from bugs in the software, because they don't.

For things that are open source, I try to use the software in-tree without
installing it. For other things, I evaluate using a virtual machine.
Seriously, I don't trust installers because all of you are bad at them.

> The problem we face is, how can we make it easy to install something, while
> still being safe and maintainable?

Google, Apple, Microsoft, et al have recommended publishing platforms (aka
"app stores") that are designed to specifically solve this problem.

For Debian and the derived, we can approach a Debian Maintainer and ask them
for help getting it into Debian. For other distributions, we can take similar
steps.

If we insist on publishing things ourselves, we can make our software really
portable: Let it live in any directory, and not touch any files. Make it easy
for the user to verify this.

If we can't do that, we can document the details: Explain all the files we
touch and why, and recommend users create separate user accounts (or
containers/virtual machines) to really protect themselves. _Try to get people
used to this level of care_ because having a positive experience with good
software with excellent documentation[1] will give you pause when faced with
anything else.

Honestly, the number of programs that want to run as root or as my user
account is terrifying, and the amount of work necessary to sandbox unknown
apps really makes me not want to bother. I know most people don't worry about
this, so purely from a "hurr hurr move fast" point of view, this isn't
anything anyone needs to worry about: `curl | bash` is good enough, and will
likely be good enough for a long time.

[1]: [http://cr.yp.to/qmail.html](http://cr.yp.to/qmail.html)

~~~
extrapickles
I don't think the installer problem will ever get better, though the portable
software idea would make things better as the application wouldn't be
scattered all over the place.

Its why I stick everything in a container now, as then I don't care what
software does to its filesystem, and I only push what directories I want it to
have into it. This also lets me run multiple versions of software when the
software does not normally support that.

------
some1else
Take-away: Always download the script and execute it locally. An attacker may
change the underlying script once you pipe it to shell.

~~~
paco3346
I appreciate the value you find in this. I too was thinking this but everyone
else seems to be find fault in what the article was pointing out.

------
rollulus
"a knowledgable user will most likely check the content first.". Really? I'm
knowledgeable in that I understand that the script may contain evil, but I'm
lazy as hell so I don't bother to check 99% of the time. Like I clone, build
and run repos without checking every SLoC.

------
amelius
Sorry, but I fail to see the difference with downloading and installing
without verifying the code. And since verifying the code is usually hard, I
don't see the significance of this article.

Please enlighten me :)

~~~
Morgawr
The difference is in the article itself. Regardless or not of the fact that
most people don't verify the scripts that they run (we should! But we don't),
the curl | bash paradigm, as explained in the article, actually makes it
possible for the attacker to provide two different types of scripts depending
on the situation.

Are we dealing with a smarter-than-average user that is downloading the script
and then running it (who knows, he might actually read it too!)? Let's serve a
legit one. Are we dealing with somebody who's just mindlessly piping curl
output into sh? We found our victim!

------
klue07
Reminds me of why I also don't copy paste text from websites directly into my
terminal.

[https://thejh.net/misc/website-terminal-copy-
paste](https://thejh.net/misc/website-terminal-copy-paste)

------
educar
> Installing software by piping from curl to bash is obviously a bad idea and
> a knowledgable user will most likely check the content first

While this is true, how is this any different from installing using apt-
get/dpkg/rpm ? I have never looked into any package I install. In fact, those
things are worse because they require root unlike curl | bash.

At the end of the day it's about trust. If you trust the author(s), you would
install it. I trust my distribution/browser/OS and I install things they want
me to install it. So, if a project suggests "curl | bash", I would do it when
I trust the project.

~~~
rlpb
It's not just about trusting that project. It's about trusting that project to
understand relevant interactions with the rest of your system, which may
include components or configurations that project's authors have never heard
of.

Developers can screw up even with the best of intentions.

I trust distributions more because interactions between components is pretty
much all they deal with. That's their specialty.

I'm a distribution developer. I know first hand how often upstream projects
screw up due to naivety, because it's my day job to fix their bugs.

~~~
educar
Don't get me wrong, I definitely don't think that curl|bash can replace
package managers. curl|bash can be used for getting software which doesn't
need the sophisticatedness (and effort) of creating packages.

------
jchiu1106
Nice exploit.

I think it can be prevented by `tee`ing bash :

$ curl -s
[https://localhost:5555/setup.bash](https://localhost:5555/setup.bash) | tee
bash sleep 3 echo "Hello there :)"

~~~
cyphar
I think you'll still hit the buffer of tee in that case, which may still cause
the problem. I'd recommend just saving the file and executing it afterwards.

------
dantillberg
I wrote up a little utility I wanted to distribute last year, and I came up
with this little script block to do it while also verifying the scripts hash:

> bash -c
> 'S="3bceab0bdc63b2dd7980161ae7d952ea821a23e693cb74961b0d41f61f557489";T="/tmp/gut.sh";set
> -e;wget -qO-
> "[https://www.tillberg.us/c/$S/gut-1.0.3.sh">$T;](https://www.tillberg.us/c/$S/gut-1.0.3.sh">$T;)
> echo "$S $T"|shasum -a256 -c-;bash $T;rm $T'

This downloads the file to /tmp/gut.sh (which hopefully works on your system),
then checks whatever file was downloaded against the hash specified (which
hopefully works on your system), then executes it, then deletes it.

I think that `shasum` is a pretty widely-available utility among Linuxes and
OSX, though not universal, but it occurred to me that it would be really
awesome to have a program that was more purpose-built to only execute shell
scripts that matched a particular hash, a la:

> curl
> [https://www.tillberg.us/c/blah/gut-1.0.3.sh](https://www.tillberg.us/c/blah/gut-1.0.3.sh)
> | shverify --sha256
> 3bceab0bdc63b2dd7980161ae7d952ea821a23e693cb74961b0d41f61f557489

Obviously, many other systems are more "secure" than curl-bashing, but curl-
bashing is _very_ convenient, and adding some sort of common utility to
support it could mitigate the most obvious security issues.

~~~
bflesch
Your approach tackles the scenario that the actual shell script
`[https://www.tillberg.us/c/blah/gut-1.0.3.sh`](https://www.tillberg.us/c/blah/gut-1.0.3.sh`)
might be untrustworthy, but the whole command including the verification hash
will most likely also be downloaded from `tillberg.us`.

If `tillberg.us` is malicious you assume that both the gut-1.0.3.sh file and
also the hash in the curl command will be changed.

So I think your idea only works when you reference files hosted with a third
party but decide to trust the guy who gives you the hash. Basically all
occurences of curl to bash piping I have seen in the wild are of the former
type, so I really don't think this helps much.

------
kevrone
What's funny is that they suggest adding a sleep in the script so that it can
be detected as a delay in byte stream delivery, but if you really want to know
if the script is being executed and by what, you could literally just have the
first line of the script make an http request with whatever meta data you
want.

------
nerdponx
Maybe this is a dumb question, but is there a difference in behavior between
"curl > tmp.sh" and "curl -o tmp.sh"?

~~~
SixSigma
not in that case

but curl -o 'file_#1.htm'
[http://example.com/[1-10].html](http://example.com/\[1-10\].html)

downloads 10 files and renames them

------
michaelmrose
I don't want to be rude but it's not luck its competency. You would derive a
lot of benefit from learning how to manage your system properly.

~~~
bpchaps
Wow. Assuming much?

I work as a linux admin professionally and work with four different distros on
a daily basis. Simply, I don't have enough time to fix every fucking issue
that happens on my operating system that prevents something from being
installed. And this happens _constantly_. Is it incompetence when the shitty
apt/yum/aur repos are broken by default where an uninstall of $ASININEPACKAGE
that needs to be reinstalled later for $DUMBSHITPACKAGE? If someone has a
solution that works for many, I will try that, then bitch when it doesn't
work, then use the competency you think I don't have to fix it.

Jackass.

~~~
dang
> _Jackass_

It's easy to understand why someone would lash out in response to having their
competence snidely negated, but it still damages the community if you do that,
so please don't.

~~~
bpchaps
Sorry about that. I'll edit my comment. If I still have your attention,
actually, would you mind deleting my account and possibly the comments? I've
seen lots of comments on here saying that it's difficult to get a response for
deletion, so raising it here might work better. I've been spending too much
time on here. :)

Edit: can't edit that post :(

~~~
dang
> _I 've seen lots of comments on here saying that it's difficult to get a
> response for deletion_

I'm surprised. Those comments must be wrong. We respond to this at
hn@ycombinator.com all the time.

HN doesn't delete accounts. We do delete specific posts that people are
worried about getting in trouble from. The reason is because deleting an
account and all its comments guts the threads that account participated in,
which is unfair to the other commenters in the discussion. The threads are
community property and we have a responsibility to preserve them. PG wrote
about this here:
[https://news.ycombinator.com/item?id=6813226](https://news.ycombinator.com/item?id=6813226).

Our approach tries to balance that concern with individual needs. Sometimes
people post things they later regret, e.g. personal or employer info. In such
cases we're happy to help by redacting or deleting the comment. But we do it
on a case-by-case basis. That way people can get their specific concerns taken
care of, but the community history remains largely intact.

We do intend to implement account renaming, which should resolve most
remaining concerns. But it isn't done yet.

~~~
bpchaps
I genuinely appreciate the response and fully agree with the reasoning. I
guess the posts I saw were some weird form of survivorship bias. Curious what
happened there, but "shit happens" sounds pretty valid. :)

Like I said earlier, I wanted to delete my account just to remove this as a
time sink, but it's a bit more than that. The minor reprimand for name calling
was a minor confirmation of it, but one of the things I've seen of late is
that many posts are _incredibly_ critical of things that often don't deserve
it. For example, a few of my posts about personal experiences with mental
health are met with strong, undeserved, and almost angry criticisms towards my
experiences. It feels overkill, and many times it validates the hostility that
those with mental health issues have when discussing these issues. HN seems to
be one of harshest sites I've seen on this subject. I mention the
reprimanding, since that's a "hard insult", rather than the "soft insults"
I've seen during mental health discussions. They both carry a similar weight,
but one (for understandable reasons) sounds sharper, so is easier to handle
and parse from someone who's dropping into the discussion. Maybe what I've
seen is just a few bad eggs, but others have shared similar experiences -
thought that may just be the converse of the bad eggs.

Is there anything that can be done to help out here? I'm not calling for any
individual solution and by no means asking for censorship of any form, but
it's something that's been on my mind recently, since unnecessary hyper-
critcal responses feel just as damaging to a community as direct insults are.
It's a stupidly difficult problem, but possibly worth mentioning. I'm sure you
guys have discussed this before, though.

(Sorry for the mild rant, and this probably isn't the best place to discuss
this, so please feel free to delete this and I can shoot an email instead.)

~~~
dang
I'm not sure what to tell you. I would love to find a way to lessen that kind
of harsh criticism on HN.

