

Curl vs Wget - helwr
http://daniel.haxx.se/docs/curl-vs-wget.html

======
statictype
>Wget requires no extra options to simply download a remote URL to a local
file, while curl requires -o or -O. However trivial, this fact is often
mentioned to me when people explain why they prefer downloading with wget.

Funny because that's _exactly_ the reason I installed macports and then wget.
It's stupidly trivial, but there you have it.

~~~
rix0r
Another stupidly trivial reason:

I can type 'wget' and then do command-V entirely with my left hand, keeping my
right hand on the mouse. Having to type 'curl' would require me to move my
hand back and forth, slowing me down.

~~~
Groxx
I love reasons like this, they make non-geeks look at us like we're insane.
I've only used curl personally, so this wouldn't be enough to switch, but it's
an excellent point.

~~~
technomancy
Whereas the true geeks look at you like you're crazy for still using crufty
old QWERTY. =)

------
1amzave
My main complaint about curl is that it prints a bunch of crap (well,
statistics technically) to stderr by default. Especially for something claimed
to be (at least in this article) "traditional unix style", that seems
undesirable.

It's annoying when I do 'curl $URL | less' and end up with garbage in my pager
that I have to scroll around a bit to make disappear.

It seems like default behavior should be more like 'curl -s -S' -- silent, but
print errors if they occur, just like...well, every other sensible command-
line tool in the world (so that's what I have curl aliased to).

wget of course prints statistics too, but it's not such "pipe-oriented" tool,
as the author points out, so this isn't as big of a deal.

~~~
tlrobinson
My version of curl only prints the progress junk to stderr if you pipe stdout
somewhere. I guess I've never had a problem since I don't use a pager usually.

BTW, how do you detect whether your program's stdout is being piped to
something other than the shell?

~~~
1amzave
> _My version of curl only prints the progress junk to stderr if you pipe
> stdout somewhere._

Hmm, good point -- I actually hadn't noticed that, and it seems to be the case
on mine as well. (Though I rarely use it without piping or stdout-redirecting
to something, so it still strikes me as inappropriate.)

> BTW, how do you detect whether your program's stdout is being piped to
> something other than the shell?

In the three languages I've used recently enough to remember:

C: int isatty(int fd); /* from unistd.h */

Python: os.isatty

Shell (bourne): [ -t 1 ] # exits 0 if stdout is a tty, 1 if not

~~~
devinj
Instead of os.isatty(sys.stdout.fileno()) (ew) you should just use
sys.stdout.isatty(). In addition to being shorter, it will work everywhere
[that Python does], not just Unix.

------
mikeytown2
What I do

    
    
      * curl for any bash scripts
      * wget for downloading via command line
    

The error codes that curl output make it very powerful when creating a script.
Something that gets run on cron & you want to know why it failed if it did.

Wget is simple, and for a shell it's what I always turn to. The only time I
would use wget in a script is if I was making some sort of simple web crawler
script.

~~~
sliverstorm
same here. I use wget to do stuff, and I use curl to do complicated/fancy
stuff. One is simple, one is powerful. I'll use simple whenever I can get away
with it.

------
henning
Probably wget's major reason for being is mirroring websites. It does this
well. Curl is a more general HTTP library with a command line front end.

And of course there are GUI frontends for wget, but it's not really intended
to be used as a library. I don't see a problem here.

------
X-Istence
I use curl for almost everything. Especially for debugging caching and
whatnot, using curl -D - -o /dev/null <http://example.net/> makes it extremely
simple to see the headers the server is sending for a certain file.

Not only that, but curl is installed standard on Mac OS X whereas wget is not,
also, most attacks that attack PHP tend to use wget to download further files
to the target system, which means that when wget is not available no extra
files are downloaded and it generally stops the exploit cold in its track as
the attacker does not try again.

~~~
vog
_> most attacks that attack PHP tend to use wget to download further files_

Nowadays, neither Wget nor Curl is required for downloading in PHP, because
functions like _file_get_contents()_ are able to handle URL arguments! So a
missing Wget protects only against script kiddies who don't know PHP very well
and copied over some outdated code snippets.

Also, not installing Wget "protects" you just from those kind of primitive
attacks to which your system shouldn't be vulnerable in the first place.

Security is achieved by sealing entry doors, not by removing tools.

~~~
X-Istence
file_get_contents on most hosts does not allow external URL's, including the
one where I am an administrator.

[http://www.php.net/manual/en/filesystem.configuration.php#in...](http://www.php.net/manual/en/filesystem.configuration.php#ini.allow-
url-fopen)

Also, installing wget is just one more way to defend a server that is hosting
multiple sites with multiple scripts all of which can not be trusted all the
time. I don't know if tomorrow a new bug is found in phpBB that allows a
remote exploit that runs as the user php runs under, if not having wget stops
one such exploit it is a win in my book.

The other thing I have in my firewall is all outgoing access is blocked for
the user that PHP runs under so that in the event something does happen and
something is executed under that user it won't be able to make an outbound
connection or get inbound connections.

Yes, security is achieved by sealing entry doors, but removing an entry door
in the first place may be even better, wget being an entry door, unfortunately
the front door is a huge gaping mess you could drive a truck through, but that
is unfortunately required for the business I am in, administrating a server
with multiple websites running various PHP scripts.

~~~
vog
_> file_get_contents on most hosts does not allow external URL's_

This is still not really a gain in security, but ...

 _> The other thing I have in my firewall is all outgoing access is blocked
for the user that PHP runs under_

... this is! And since you do that, there is no need for strange measures like
providing Curl but not Wget.

Wget, Curl and allow_url_fopen are just tools to access the network API. So
you need to block that API, rather than blocking some tools which merely use
that API.

Although I fully agree that it improves security if you remove all _services_
that aren't needed, I don't think the same holds for _tools_. (except if they
are SUID binaries, but such a program is in fact more a service than a tool)

~~~
DougBTX
Good points all round, X-Istence is suggesting defense in depth, and you (vog)
suggest making sure the outer perimeter is secure. Both good ideas - even if
you fill the hall with barbed wire, you should make sure to lock the front
door. Beyond that, it's a trade off.

------
dlsspy
"libcurl does not put any restrictions on the program that uses the library."

I wonder how much of the activity could be attributed to such things.

------
Groxx
That's a nice breakdown... I've been a curl user, and never really knew what
the major differences were. And it doesn't seem overly biased to me. Many
thanks to the author!

------
barrkel
I find wget is almost always what you want, unless you're interested in
development details like headers. If I want output to a pipe, I use wget -O -.

In scripts, I use wget -o <log-file> (or -a) so that I can tail -f to keep
track of it. I don't see off hand what curl's log file support is, so I can't
comment too deeply on it.

------
pibefision
Does anyone knows how to mirror a site using wget or curl, and get all the
assets? (specially images linked on stylesheets).

~~~
Tichy
Some time ago I filed a bug report because of the stylesheet images. So it
still isn't fixed? It's simply a bug.

I have tried and liked pavuk in the past, don't know if it supports stylesheet
images.

------
cowsandmilk
lwp-download should be in this comparison. I've found lots of random cases
where it does stuff right and the others don't.

~~~
godDLL
As should aria2. But that would make it a completely different type of an
article, more like a Wikipedia sub-section. We already have Wikipedia.

------
whyme
When downloading via HTTPS curl seems to barf error codes often so I try wget
first, and use curl only if wget doesn't work or curl is easier (which isn't
often).

------
mjs
wget has a better-designed command line API and documentation, although its
inability to PUT or POST is rather unhelpful. For example, curl uses --include
to output full headers (wget uses --save-headers) and, more inexplicably,
--request to change the request method, even though --method is available.

~~~
barrkel
Use --post-data or --post-file to POST with wget.

~~~
mjs
\--post-* is pretty sensible for POST. Unfortunately wget doesn't do PUT or
DELETE.

------
ighost
kinda banal if you ask me. it's two simple tools that do approximately the
same thing. even grep vs ack would be more interesting.

no offense to the authors of course, they did a great job in both cases.

~~~
sorbits
As someone who have always used curl and wondered why I often see wget quoted
in READMEs (as how to obtain some remote file), I quite enjoyed reading the
write-up.

My take-away was that a) I am not missing out by staying with curl and b) the
use of wget is likely due to being part of GNU.

~~~
ZeroGravitas
You did notice this was written by curl's author? I only noticed after I read
it and without that knowledge thought it was rather a pathetically biased
article.

But you can't expect developers to be impartial about their own code and it
was my fault for not noticing the disclaimer the first time. It reads
completely different with that framing in mind, but still if I was looking for
the definitive reason that one tool was recommended over another I'd consider
the point of view of the developers as a starting point rather than an open
and shut case.

