

Stop Using FTP - bdotdub
http://stevenf.com/archive/dont-use-ftp.php

======
SwellJoe
Note that "SFTP" has no single meaning. Depending on the client, it may mean
"FTP encrypted with SSL" or "FTP with encryption extensions of one of a couple
of types invented independently by FTP server authors over the years" or "SSH
File Transfer Protocol" (which is actually the real meaning of the
abbreviation, and was in use for several years before secure FTP began to be
common).

Historically, people tried to convince folks to call it FTPS (for FTP over
SSL, which is the most popular form of secured FTP), but that seems to have
never taken off. Unfortunate, since SFTP means too many incompatible things.

It's stabilized somewhat in recent years, and usually clients will do the
right thing--and the most common meaning of SFTP has come to be "FTP encrypted
with SSL" (which is actually one of the weaker forms of security for FTP--but
it's got some semblance of "standardization", and works in lots of clients and
servers, so that makes it superior). And, of course, there is no RFC defining
secure FTP, so it's a de facto "standard" which can lead to buggy
implementations and a lot of finger pointing--server says client is wrong,
client says server is wrong, user says "I give up" and uses FTP, instead.

~~~
nuclear_eclipse
On that note, from a Linux system, what does `sftp` use on the command line? I
had always assumed that SFTP was always and only SSH transfers, just like
using `scp`. Is this wrong of me to think this way?

~~~
SwellJoe
_I had always assumed that SFTP was always and only SSH transfers, just like
using `scp`. Is this wrong of me to think this way?_

No, this is absolutely correct. But don't be surprised if someone
misinterprets you.

~~~
tptacek
I'm not sure I agree with you 100% on your police work there, Lou. SFTP is its
own protocol; it runs over SSH, but is substantially more complicated than
"scp". "sftp" is not just user interface, like scp is.

~~~
SwellJoe
I took his comment to mean, "I've always understood SFTP to be the FTP
protocol that runs over SSH", and so I replied that his understanding was
correct.

I didn't consider him saying "like using scp" to mean he thinks "it _is_ scp".
Rather, I took it to mean, "it uses SSH to copy files, like scp".

But, I can see how one could interpret it that way. It was not my intent.

I'm staying the heck of out of the pedantry business in the future. This has
been a ridiculous bundle of threads all spawned just because I had a bee in my
bonnet about how the term SFTP is confusingly misused by lots of folks. I
don't even use any of these "FTP" protocols for anything (I use rsync or SVN
for website data, and scp for one-off file copies--I haven't used an FTP GUI
since I owned an Amiga). I only ever touch FTP for testing problems our users
report, and then always using lftp on the command line. I've got no dog in
this race...

------
gaius
You know, back in the day, there was an idea called "opportunistic
encryption". How it worked was, you would put a FreeBSD box running S/WAN at
the edge of your network, and if it detected a similar box at the other end,
they would seamlessly encrypt your stream (and if not, just let it straight
through). Now, if this idea had gotten the traction it deserved, we could all
happily be using telnet, FTP, whatever, rather than _everyone_ having to be
aware of encryption, your SA would just sort it once, for everyone, and they'd
keep on using familiar tools.

~~~
notauser
Opportunistic encryption is generally a good idea, but as it has no feedback
method you can't really be certain if it is working or not. The seamlessness
works both ways.

~~~
gaius
Right, but the current situation is silly. For example right now if I want to
connect into work, I fire up the Cisco VPN client, _then_ I ssh from my Mac to
whatever Unix box I want to work on there. I'm not sure that "double
encryption" really buys us anything here. Unless someone's actually in our
datacentre tapping the wire coming out of the VPN host, in which case we've
bigger problems...

~~~
devicenull
It's not really for the double encryption. It has a big advantage in that you
don't need to open SSH up to the internet, only to your VPN server. Even
though you may have perfectly secure passwords, why expose SSH to the internet
if you don't have to?

------
tptacek
SFTP is its own debacle. It's a handy alternative to "scp" when you don't want
to open a second connection to grovel through "find" on the target system, but
it's even less firewall-friendly than FTP: you can't enable users to grab
files with SFTP without _also_ enabling them to SSH to remote hosts.

Most enterprises _don't_ allow outbound SSH through their firewalls, in part
because SSH is its own slippery slope: if you're allowing SSH, you're allowing
people to build VPN tunnels.

The real-world operational problem that FTP solves is upload. There is no
download application that HTTP doesn't solve better than FTP. If Apache
shipped with a mod_upload with a reasonable configuration system and a simple,
workable web interface for specifying what to put where, FTP could be laid to
rest.

~~~
LogicHoleFlaw
I have seen enterprise systems where sftp access is granted to third parties
by restricting the login shell severely to allow only sftp uploads. Is there a
problem with that approach? SOX / PCI is making the 'secure upload problem'
more pressing daily.

~~~
tptacek
Yes, for instance: X11 and port forwarding.

The bigger problem is, it's just hard to know what else you're allowing once
you let someone "only" access SFTP via SSH.

Most of the BigCo's I work with use "Secure File Transfer" software to solve
this problem, which are just glorified upload CGI scripts under SSL. That's
the best solution, I think.

------
deathbyzen
I'd definitely leave age out of the argument that FTP is bad seeing as how
plenty of people use old programming languages, old OSes, and old software for
the simple fact that it "just works."

------
aaronblohowiak
this is a matter of right tool for the job. for important stuff, everything
should happen over a secure tunnel (ssl, ssh port tunneling or just using
scp.) ftp is good for public file download/drop box where you want to easily
support resume.

------
emmett
Almost all his arguments apply equally to HTTP.

~~~
anthonyrubin
\- Unless tunneled over a secure socket, FTP is 100% insecure.

This is true for HTTP.

\- The spec defines no way of setting the modification dates/times of files.

The HTTP RFC defines the Last-Modified header.

\- The spec defines no reliable way of determining the string encoding used
for filenames.

The HTTP RFC defines several headers related to encoding.

\- FTP was designed to be used interactively by a human sitting at a terminal,
not by a GUI application working on the human's behalf. The spec doesn't even
define the output format that should be used for directory listings.

This problem doesn't translate exactly to HTTP as it isn't used in the same
manner. Directory listings are typically sent as HTML and a user clicks a link
which all browsers are able to handle trivially. WebDAV adds features similar
to FTP, but it is much more standardized.

\- The spec defines no way of dealing with file metadata, such as Unix
permissions, owners, and groups.

There are HTTP error codes which indicate lack of permission, but there are no
RFC-defined headers for these things. I believe WebDAV does define a method to
deal with them.

\- FTP requires a minimum of two socket connections to transfer a file: the
control connection, which is established first, and then data connections
which are created and destroyed every time you transfer a single file, or
request a directory listing.

HTTP has no such requirement. Keep-alive prevents sockets from being closed
and the same connection can be used to get a directory listing and then
transfer a file. There are no separate data and control connections.

\- FTP is not friendly with firewalls. Because it constantly needs to
establish new connections, this has led us to "passive mode" which might as
well be black magic as far as most people are concerned.

This is FTP's biggest flaw and this problem absolutely does not exist with
HTTP. Once it is encrypted FTP is impossible to handle in a reasonable manner
with a firewall.

In active mode FTP creates one connection to a well-known port for the control
channel (21). For the data connection the server attempts to establish a
connection from a well-known port (20) to a high-numbered port on the client.
This obviously doesn't work well with a firewall in front of the client.

Passive mode modifies this arrangement so that the data channel comes from the
client to the server. The problem is the ports used. On both ends the port
will be a high-numbered. This requires allowing connections from any high-
numbered port (withing a large range) to any high-numbered port. Some
firewalls understand the FTP protocol and are able to see the port that is to
be used; the server gives the client this port with a command in the control
connection. The firewall can then allow that specific connection. Once the
control connection is encrypted this is impossible.

Now then. How do his arguments apply equally to HTTP?

~~~
emmett
"Almost all" might have been hyperbole, but "many" is certainly true.

On the other hand, HTTP is clearly far superior to FTP, especially if you
include extensions like WebDAV and the power of custom headers.

I was just pointing out, insecure formats that don't define how to do listings
or set mod times or deal with Unix permissions can be quite good. FTP sucks
mostly because of the ports/multiple connection issue, as you pointed out.

------
tx
Well, I'd gladly switch to sftp, but dumb-ass OSX and Windows don't understand
it out of the box and most users are unaware of more powerful (and easier to
use) alternative OS, which leaves us stuck with this commercialized
mediocrity.

Why wouldn't we also stop using Apple Finder, FAT, IE6, non-UTF encodings and
bunch of other legacy crap? Ohh-ho-ho... so many topics to blog about...

~~~
froo
I'd say that the reason why people aren't switching so readily is actually a
function of the human condition moreso than any technical reason.

Looking at other industries, like health care, people are more content in
spending effort/money on "miracle cures" than worrying about taking steps
towards preventing these problems from happening.

The exact same thing happens in say, the Automotive industry - a large
percentage of consumers dont take active measures towards ensuring their
vehicle is in good running order, and are more content with dealing with the
consequences.

The same thing seems to happen when talking about computer security. I dont
think it's because people dont care about computer security, they're just more
ready to accept the consequences rather than taking preventative steps.

It's a real damn shame too.

~~~
tx
What? It's MUCH simplier than than: I can't send a Windows/OSX user an email
with an SFTP version of this line:

ftp://server.com/report.pdf

It's that simple.

~~~
tptacek
Why would you ever send ftp://anything when you could instead send
<http://anything>?

~~~
tx
Yep, that's a popular response I get from OSX people: if Apple can't do
something, that's because "you don't need it".

~~~
tptacek
Huh?

~~~
tx
What's up with the sound? You just took a shit and can't reach your ass?

------
kylec
I'd like to use SFTP or scp for everything, but I've discovered that it's
considerably slower than regular FTP. I wish that there was a way of
encrypting the login but sending everything else via plain FTP for speed.

~~~
SwellJoe
You can do that with WebDAV and Digest authentication. I don't actually know
that it's comparable in speed, though.

I'd also question the speed argument...for large files, SSH can perform
compression and make the transfer faster than FTP.

~~~
kylec
Thanks for the compression tip! My extremely informal testing produced the
following results taken by copying a random binary file I had to another
computer:

SSH: 2.0MB/s

SSH w/ compression: 2.5MB/s

FTP: 2.8MB/s

While SSH does not approach the speed of FTP it does come close, perhaps close
enough to finally ditch FTP altogether in most situations.

~~~
gaius
It depends to a certain extent on what cipher you use. Try Blowfish, AES, etc
for the kind of data you move and see which one is better. We aren't talking
by much, mind, but every little helps.

------
kmt
sftp is not ideal; looks like the author is not aware that ssh (which is what
sftp uses) is rather slow on WAN due to buffer issues:

<http://www.psc.edu/networking/projects/hpn-ssh/>

[http://www.psc.edu/networking/projects/hpn-
ssh/papers/i2mm-h...](http://www.psc.edu/networking/projects/hpn-
ssh/papers/i2mm-hpn-ssh.ppt)

------
pon
I think using encryption limits the size of the files to about 2 GB in current
versions. It is too little for DVD images.

~~~
SwellJoe
Simply not true.

The openssh family of tools have had LFS support for many years. I've used scp
several times to copy huge files...and our products have scp as a filesystem
backup target, which means it deals with archives that are dozens or hundreds
of GBs in size on a daily basis.

