
Unix Wildcards Gone Wild - 0x0
http://www.defensecode.com/public/DefenseCode_Unix_WildCards_Gone_Wild.txt
======
AceJohnny2
David A. Wheeler, FOSS (and occasionally security) luminary, who also happens
to be creator of the popular sloccount tool, has an excellent page that covers
this topic and how to use paths safely and portably in shell scripts (spoiler:
it's _hard_ ) :

[http://www.dwheeler.com/essays/filenames-in-
shell.html](http://www.dwheeler.com/essays/filenames-in-shell.html)

I strongly recommend his many other essays to HN readers:
[http://www.dwheeler.com/](http://www.dwheeler.com/)

Edit: a simple way to avoid these problems is to prepend the wildcard with ./
(so globbed files won't start with - or -- but with the path ./) and on GNU
systems put -- before the wildcard, telling the tool that following arguments
are _not_ options.

~~~
teddyh
The double dash -- to stop option parsing is not a GNU thing – it comes from
the POSIX standard.

~~~
AceJohnny2
Thanks! I had always figured that, along with long options, they were a GNU
thing.

~~~
_delirium
There's some digging here:
[http://unix.stackexchange.com/questions/147143/when-and-
how-...](http://unix.stackexchange.com/questions/147143/when-and-how-was-the-
double-dash-introduced-as-an-end-of-options-delimiter)

Looks like the convention was introduced as part of getopt in AT&T Unix System
III (1980), though initially only in a handful of places. Then POSIX adopted
it as standard for all utilities sometime later.

------
sebcat
"I'm not sure how to put it mildly, but I think you might have been scooped on
this some 1-2 decades ago..."

[http://seclists.org/fulldisclosure/2014/Jun/136](http://seclists.org/fulldisclosure/2014/Jun/136)

------
catern
I really wonder who this person asked, that was an "old-school Unix admin",
that didn't know of this attack. This article also doesn't mention the
countermeasure which is available in every utility I know of: the -- argument,
which disables parsing of all further arguments and treats them just as
filenames.

------
raldi
Even if you're using a tool that doesn't support "\--" you can just use "./*"
and everything will be fine.

------
evacchi
Many of these gotchas have been known for quite a while. I suggest, for people
who still don't know it, to read the UNIX-HATERS handbook

\- homepage [http://homes.cs.washington.edu/~weise/unix-
haters.html](http://homes.cs.washington.edu/~weise/unix-haters.html)

\- working download link [http://richard.esplins.org/static/downloads/unix-
haters-hand...](http://richard.esplins.org/static/downloads/unix-haters-
handbook.pdf)

A lot of it is outdated, and yet many things are still incredibly relevant

------
JadeNB
Many people are recommending the '\--' option-terminating option. Note David
Wheeler's caution (in the essay already linked by AceJohnny2 at
[https://news.ycombinator.com/item?id=8190208](https://news.ycombinator.com/item?id=8190208))
about why this is not an all-purpose solution:
[http://www.dwheeler.com/essays/filenames-in-
shell.html#dashd...](http://www.dwheeler.com/essays/filenames-in-
shell.html#dashdash).

~~~
claudius
The reasons given there aren’t really any good:

    
    
      1. For “--” to work, all maintainers would have to faithfully use “--” in
         practically every command invocation. That just doesn’t happen in real
         life,  even after decades  of people trying.  People forget it all the
         time;  no one is that consistent,  especially since code seems to work
         without it. Very few commands require it, after all.
    

So because other people may or may not forget it, I shouldn’t use it in my
scripts/day-to-day usage? That’s about as silly as saying that, because other
people will write unreadable code anyway I shouldn’t bother with comments,
short functions or sensible variable names.

    
    
      2. You can’t do  it anyway,  even if you  were perfectly consistent; many
         programs and commands do not support “--”.  POSIX even explicitly for-
         bids echo from  supporting “--”,  and echo must support “-n”  (and GNU
         coreutils echo supports other options too).
    

This is a problem, but iff you have to use _echo_ for some reason. _printf_
works nearly equally well and supports -- just fine. I may have read somewhere
that using printf is actually advocated nowadays, but I’m not sure where and
why.

~~~
JadeNB
> So because other people may or may not forget it, I shouldn’t use it in my
> scripts/day-to-day usage?

He specifically mentions that that is not his point, but rather that he is
arguing against exactly the sort of "just use '\--'" response that one can
find in this post:

> _Do_ feel free to use “--” between options and pathnames, when you can do
> it, as an _additional_ protective measure. But using “--” as your primary
> (or only) mechanism for dash-prefixed filenames is bad idea.

------
squidsoup
The excellent Shellcheck static analysis tool will save you from many of these
potential pitfalls:

^-- SC2035: Use ./* so names with dashes won't become options.

[http://www.shellcheck.net/about.html](http://www.shellcheck.net/about.html)

------
mschuster91
I have worked with Linux systems since I was able to write and have heard
about this the first time. Thanks HN \o/

------
jingo

       echo rm *
    
       echo chown -R nobody:nobody *.php
    
       echo chmod 000 *
    
       echo tar cvvf archive.tar *
    
       echo tar cf archive.tar *
    
       echo rsync -t *.c foo:src
    

This article starts with the premise that the person executing the command has
no idea what files are in the current working directory. That is itself a more
serious problem than the behaviour of wildcards.

Later in the article we learn that it also assumes GNU utilities. That is a
second problem (IMO), and arguably also one more serious than the behaviour of
wildcards. GNU userland and unneeded complexity (e.g. more features than any
user will ever use) are practically synonymous.

Then there is the peculiar assumption that someone can place arbitrary files
beginning with - or -- on this system. That itself is a far more serious
problem than the behaviour of wildcards; I would say with that capability it
is more or less "game over". In BSD you have, at the very least, mtree. How
does the Linux user know she isn't executing some substituted executable?

Moreover, if caution was important to the hypothetical user in the examples, I
think they would be in the form

    
    
       /path/to/program *

~~~
jordigh
> GNU userland and unneeded complexity (e.g. more features than any user will
> ever use) are practically synonymous.

Ahhhh, the memories. These accusations bring me back to the early 1990's.
Remember? Remember how it was? Oh, boy, how did we all get so old? To be young
and running System V again...

And still today, whenever I'm faced on a system without GNU utilities, I find
myself installing them to get what seems to me like basic functionality. We
have never changed, have we?

~~~
jingo
To be young and running VMS again...

------
leni536
I remember when I used _pdfimages_ to extract images from a pdf file and I
didn't read the manual beforehand. It turned out that you should call it like
_pdfimages <pdf_file> <prefix>_ and if you don't specify a prefix it generates
filenames in the form of -img001, -img002, ... (or something like that). I had
a hard time deleting those images.

------
lobster_johnson
Tangentially, anyone know how to make zsh less greedy about parsing wildcards?
Something like this will fail with "no files matched", and the command won't
run:

    
    
        rsync example.com:/foo/* .
    

My workaround is to quote the argument, but it's annoying.

~~~
dijit
escape it, since you know the shell is going to be greedy about things.

rsync accepts globbing it's not a shell expansion that makes it work.

    
    
         rsync example.com:/foo/\* .

~~~
lobster_johnson
Well, the problem is that "example.com:/foo/*" isn't actually a local path
that can ever be globbed by zsh alone -- it's an rsync/ssh/scp-style remote
path, yet zsh interprets it as such. Not sure why. I think the solution is
make sure zsh ignores arguments containing colons, but I don't know what the
config option is for that.

Edit: Just remembered that zsh's over-eager globbing also fails with git --
eg., "HEAD^" must be quoted.

------
dap
Good summary of surprising behavior. You can work around a lot of these issues
using "\--" as an argument before you use any wildcards. This tells most
commands to stop processing options and treat the rest of the arguments as
files (or whatever other non-option arguments the command takes). That's
getopt(3)'s behavior[0]. For example, "rm -- *" will not have the problem
where directories are removed if there's an entry called "-rf" in the
directory.

[0]
[http://pubs.opengroup.org/onlinepubs/9699919799/functions/ge...](http://pubs.opengroup.org/onlinepubs/9699919799/functions/getopt.html)

~~~
mikeash
A better workaround is to prefix your wildcarded arguments with ./ as this
will work with all commands. For example, rm ./* will safely remove files
regardless of how they're named.

------
X-Istence
Thankfully on BSD the options are passed before the file names.

For example chown username:username files directory -R

Doesn't actually work. You have to move the -R to before the usernames. chown
-R username:username files directory.

Same thing with rm.

------
pdkl95
This is why I not only use "\--" everywhere, I also religiously use full
quoting of "${vars[@]}" and options like mv(1)'s --no-clobber when
appropriate. Even without the security concerns, this kind of "least
privilege" approach can help prevent a lot of really-annoying bugs.

That said, I going to have to check a few scripts for that chmod attack (or
similar) - I think I've seen that type of attack before, but I must have
forgotten about it... _sigh_

------
emmelaich
This is one of the reasons sudo should (by default) only allow a whitelist of
built-in commands to be run with wildcards.

Somewhat like sudoedit.

This is of course for the corporate case of a less privileged user performing
a certain task at elevated privileges. Not for the more common use of sudo
(these days) of people managing their own personal machines.

~~~
mzs
It's the shell that expands the wildcards which are passed to sudo as
arguments.

------
ganessh
This is a dumb question.

Can files be created with "-" in Unix? I am using a Linux system and I am not
able to do so using vim or touch commands.

~~~
annnnd
Try this:

touch -- -asdf

~~~
ganessh
That works. Thanks :)

------
hartator
I think if you have the ability to create new files on a remote host, you are
already compromised. No need to wait for an Admin mistake.

~~~
derekp7
There are hosts with multiple users on them, who have some level of write
access to somewhere on the filesystem. After all, Unix is a multi-user system,
so it is not heard of to have multiple users on it. That being said, this
article is just stating to be careful of wildcards when you are sitting in a
user-owned (or user-writable, such as /tmp) directory.

------
cLeEOGPw
Was this bug already fixed in OpenBSD?

------
ZoF
There's a reason this isn't talked about, it's not an actual legitimate/common
vector to compromise a server.

~~~
Maken
Yes, because no user-uploaded filename will ever be parsed by a maintenance
script in a server.

~~~
ninkendo
Not by any sane person, no. Why are you letting people upload and name their
own files on your server? Should we be posting articles about the
vulnerabilities in the finger daemon or Solaris 8's NIS implementation, while
we're at it?

It seems like this article is aiming towards shared servers where you actually
allow shell login to "untrusted" users, which IMO is a relic of days long
past, that only really persists at (maybe) universities. Hell, even at the
university I last worked at 6 years ago we just gave everybody their own VM.
And nowadays I wouldn't even need to give them that... students can download
and run a vagrant environment on their personal machine in like 2 commands.

It's not to say there's no audience for articles like this... I'm sure there's
plenty of environments out there that still follow the multi-user server model
from the 1970's, and I certainly pity anyone who has to administer those types
of systems. But it certainly should be no surprise to _anyone_ that there's a
lot of malicious things you can do if you have shell access to a system (or to
your point, the ability to upload arbitrarily-named files with arbitrary
content. _shudder_ )

~~~
6c6db4ab
Lets say you have web server, this web server have web application that allows
users to upload files. Web application is sane and doesn't allow path
traversal and have proper .htaccess inside of the upload directory.

Now all you need is a user who kindly requests for a copy of uploaded files.
If you're not aware of this issue (and you must be "actively aware" i.e. watch
out for it all the time) you can do something that shouldn't be possible.

~~~
ninkendo
I'm a bit late replying, but I wouldn't consider an application that allows
users to upload files, _and pick their names_ to be a sane application. Do you
think imgur (as an example) lets users name their files? What about stuff like
the defunct megaupload or other file sharing sites? You get to pick filenames
but that's really metadata... the URLs and (presumably) the underlying file
storage structure are database-driven.

