11 ms·

FTP Must Die

174 points by werdnanoslen 15y ago

kree10 15y ago
Text-only Google cache: http://webcache.googleusercontent.com/search?q=cache:6jeyroOZZnsJ:mywiki.wooledge.org/FtpMustDie&hl=en&gl=us&strip=1 http://webcache.googleusercontent.com/search?q=cache:6jeyroO...
dredmorbius 15y ago
Coral cache: http://mywiki.wooledge.org.nyud.net/FtpMustDie http://mywiki.wooledge.org.nyud.net/FtpMustDie
evmar 15y ago
You might be amused to browse Chrome's implementation of FTP, which has separate modules for each of the different types of FTP servers you might encounter, as each emits the "ls -l" output in a different way and that is (I guess) the only way to get file sizes. http://git.chromium.org/gitweb/?p=chromium.git;a=tree;f=net/ftp;h=a9ccad9efc957926f20f8825a4ffc95c24673729;hb=HEAD http://git.chromium.org/gitweb/?p=chromium.git;a=tree;f=net/... see all the "directory listing" files. The unit tests are full of scary cases like // Tests for "ls -l" style listing in Russian locale (note the swapped // parts order: the day of month is the first, before month).
- toyg 15y ago
  That's actually the standard European order. Testing the Russian locale might be a scary proposition, but certainly not for this specific reason.
  - sjwright 15y ago
    It's not just the standard European order, it's the standard order for nearly everywhere on the planet except: > MDY: Belize, USA, parts of Canada, Philippines, Saudi Arabia > YMD: Japan, China, Iran, small bits of Europe > DMY: Probably 3/4 of the planet's land surface That said, YMD should make the most sense (and is most consistent with the universally accepted HMS time format). I try to use YMD wherever I can.
    - reitzensteinm 15y ago
      YMD is logical, because it puts the most significant number on the left. But both DMY and HMS are consistent in another way - the information is ordered from most to least important for every day access, from left to right. A vast majority of dates on advertising, tickets, timetables etc actually leave off the year, unless it's ambiguous. Similarly, one might say something like 'on the 4th', which implies 'the 4th of this month' (or the 4th of next month, if applicable). This is the reverse of HMS, where the most significant number (the hour) is most important, and seconds are basically ignored day to day. MDY, on the other hand, should be taken out the back and shot (IMO).
      - wvenable 15y ago
        > the information is ordered from most to least important for every day access I argue that MDY is ordered from most to least important for every day access. Year is easily implied so it is last so that leaves month first and then day. You'd be surprised how many Canadians, which is officially DMY, actually prefer MDY.
      - eslaught 15y ago
        In my opinion YMD is the most useful because it is the most visually distinct from the other two. A date like January 2nd 2012, when arranged in either MDY or DMY format, is difficult to distinguish from February 1st. YMD on the other hand is obvious, independent of what the reader is used to.
        trebor 15y ago
        MDY is syntactically correct English: January 30th, 2012. Or in older form (where it would have come from): "Dated: January the 30th, in the year of our Lord 2012." Even the US Constitution is dated this way. I strongly prefer this way because it makes the most sense in the English language. For programmatic sorting YMD makes sense because is automatically sorts by Y, M, D, if compared as a string. (EG: on a filesystem.)
      - sunchild 15y ago
  - dchest 15y ago
    The comment says "Russian" because they really test the month name in Russian: "23 май 2011". -rwxrwxr-x 1 ftp ftp 123 23 \xd0\xbc\xd0\xb0\xd0\xb9 2011 test The note about day, month order is useful because all other tests have "%b %e" format and rows are full of other numbers: drwxr-xr-x1732 266 111 90112 Jun 21 2001 .rda_2 http://git.chromium.org/gitweb/?p=chromium.git;a=blob;f=net/ftp/ftp_directory_listing_parser_ls_unittest.cc;h=b14d5c075a9fa6d65d58377139c7f7aadba5e9fa;hb=HEAD#l103 http://git.chromium.org/gitweb/?p=chromium.git;a=blob;f=net/...
TwoBit 15y ago
Time and time again I resort to FTP to transfer files between computers that won't talk to each other via system-provided file systems. Mac<->Windows file sharing is so unreliable that I don't even try to use the system any more and just go straight to FTP. But yeah I hate the protocol itself.
- wollw 15y ago
  Have you tried using SCP? I don't know if Windows comes with a client but a quick search turned up a few options. It is my goto method for transferring files these days but I admittedly don't use Windows.
  - eCa 15y ago
    WinSCP has served me well over the years: http://www.winscp.net/ http://www.winscp.net/
  - ecaron 15y ago
    I'd love to know the transfer speed differences between SCP, SFTP and FTP - because when I can view both ends of the wire then security is less of a concern than transfer time.
    - kronusaturn 15y ago
      The difference is negligible when both machines are fast enough to encrypt/decrypt at wire speed, otherwise FTP is faster.
    - astrodust 15y ago
      The overhead of encryption is marginal and the advantage of being able to compress the stream is considerable. scp and rsync, which is a fantastic companion, allows on-the-fly compression of content to get truly impossible speeds over the wire.
      - zoobert 15y ago
        It is marginal if you want to transfer a file from time to time but if you are in a business where you transfer thousands to millions of files everyday it makes a big difference. FTP is still used because its interface is simple and well-known: good old file-directory paradigm that everybody knows. It is also used because it runs on every platform. I am in favour of having a new protocol but it should: - be faster than FTP (would have to use UDP instead of TCP here). - Firewall friendly. For that, you could use one port to do everything and forget about the data connection (Is it feasible with UDP as you would need to control the flow ?) - Being FTP compliant from the interface point of view. You should be able to replace your ftp client with your fastTP protocol and everything would run. - Add optional strong security. Possibility to encrypt or not the data stream and encrypt the password negotiation. If we have this and it is marketed well then it could replace FTP.
omh 15y ago
FTP is definitely outdated, and there are better uses in many circumstances. But I find some of these criticisms a bit odd. The client listening was largely solved by 'passive' mode, and just about every server and client supports this now. The firewall and NAT interaction is awkward, but most modern firewalls can deal with this automatically (as long as there's no SSL involved) And yes, the RFC is 20 years old. But so are many RFCs for long-established veteran protocols that we use all over the net. In many cases I'd be happy to see FTP replaced (all those anonymous FTP servers may as well be HTTP now), but it's really not that bad.
- tptacek 15y ago
  PASV mode came with its own security drama, and doesn't address the core issue with the (absolutely pointless) multiple-connection model of FTP. And so, yes, firewalls and NATs "interact" with FTP --- because they all had to be hacked specifically to deal with the FTP protocol, which moots your rebuttal --- but in doing so they create additional security issues, like the NAT pinning stuff Samy Kamkar posted last year. It shouldn't be necessary for middleboxes to hack in support for protocols by in-place editing TCP streams and dynamically changing filter/translation rules based on stuff that happens inside of connections. But, thanks to FTP, they do have to do that. And for what? FTP is even on its own terms a terrible file transfer protocol! For instance, look at how it handles (or, commonly, doesn't handle) file listings. FTP is an anachronism. It has no more reason to exist today than TFTP --- both were designed to make up for constraints in client software that simply no longer exist anywhere.
  - justincormack 15y ago
    Have used tftp for net booting, those legacy bioses cant do anything else, as it really is trivial...
    - tptacek 15y ago
      Has there been a piece of equipment shipped in the last 10 years that legitimately couldn't get a TCP running to bootstrap itself? That's why TFTP exists: to avoid the need for TCP.
charliesome 15y ago
> The LIST command is Maximum Bullshit. And I'm not even touching the subject of opening a data connection to send the list itself. Did you know that the specification doesn't tell anything about WHAT should be sent as the list? Well it's a good thing that just about every FTP server in existence supports the MLSD command then.
pygorex 15y ago
FTP is like religion: poorly designed at inception, implemented a thousand different ways and accepted as normal only because it's ubiquitous.
- sjwright 15y ago
  Except unlike religion, FTP began with a legitimate use-case scenario.
  - rplnt 15y ago
    Don't know about all religions but some began with clear use-cases. To retain (or even achieve) power over people. For example The Old Testament was written/compiled for this very reason[1]. 1. http://books.google.com/books?id=iS7rQwAACAAJ http://books.google.com/books?id=iS7rQwAACAAJ
- Nelson69 15y ago
  FTP is kind of dying out though. Now the religion of SMTP.. I often wonder how the spam situation would look if we redid SMTP and required certified TLS. What's worse, we still use SMTP for real stuff every day, I get emails from Etrade, Fidelity and others that I simply don't want flying through cyberspace decrypted even if they don't contain real information.
- gaius 15y ago
  The people who designed FTP were not stupid: they did the best they could operating under constraints that kids who have grown up with all the serious work of implementing the Internet already done for them cannot imagine. It's easy, with decades of hindsight, to claim you could do better. I bet you couldn't.
thristian 15y ago
It's been a long time since I've been involved with transferring files to and from clients, but even five years or so ago I recall enabling IIS's WebDAV mode in preference to handling customer-support calls about getting FTP working. Of course, WebDAV is still a ball of horror, even if it's not as bad as FTP, so please let's all stick to SFTP from now on?
- eli 15y ago
  So you had XP boxes connecting via WebDAV? I think I'd take FTP over having to support that.
  - thristian 15y ago
    Indeed. It worked pretty well, actually - in the standard Windows Explorer 'mount network drive' dialog, you could type an HTTP URL and get an ordinary-looking Explorer window and drag files to and fro.
alecco 15y ago
Browser HTTP file upload is still terrible for file transfer. One of the last bastions of Flash, regretfully.
- tptacek 15y ago
  Browsers are terrible at file upload. The weakness here isn't really in the protocol.
  - alecco 15y ago
    That's why I said browsers. But only recently with HTML5 XHR file uploads we could have progress and status reported in-page (instead of poorly in the status bar, if any). And it's still not good enough and definitely not widely available.
    - tptacek 15y ago
      But the choice isn't between "browsers doing FTP" and "browsers doing HTTP"; it's between special-purpose FTP software and (somewhat hypothetical, if you exclude Transmit) special-purpose HTTP software.
    - axiak 15y ago
      Huh? As long as you have XHR and JS, you can do file upload progress bars. Gmail has been doing it for years, and I've certainly had implementations going back to 2007. Now, if the upload is failed for whatever reason (i.e. the file is rejected in-progress), then the upload failure mode is terrible unless you resort to the usual iframe hackery (Connection Reset screen). HTML5 XHR file upload is certainly a cleaner solution and something everybody has wanted, but it's certainly not needed for progress and status reported in-page without flash.
      - alecco 15y ago
        OK. Small nitpick, Gmail used a Flash uploader for many years, at least until 2009: http://ajaxian.com/archives/multi-file-upload-in-the-flickr-and-gmail-house http://ajaxian.com/archives/multi-file-upload-in-the-flickr-...
sehugg 15y ago
I don't think FTP must die so much as RSYNC MUST LIVE. (Seriously, people forget how awesome rsync is. And it tunnels/compresses nicely over SSH)
- kree10 15y ago
  Also don't forget that rsync does not overwrite a remote file until it's completely uploaded. Compare to ftp, sftp and scp, which overwrite the file from the start, possibly breaking your site during the transfer (or definitely breaking your site if the transfer dies part-way through).
  - plq 15y ago
    > Also don't forget that rsync does not overwrite a remote file until it's completely uploaded. which is really troublesome when you're transferring a big file and the target doesn't have enough space for a second copy of that particular file. I, for one, would prefer that "feature" to be optional.
    - kree10 15y ago
      Take a look at rsync's "--inplace" option. I think this does what you want.
nviennot 15y ago
There is something pretty cool that one can do with the FTP protocol: FXP transfers to perform an inter-server file transfer: https://en.wikipedia.org/wiki/File_eXchange_Protocol https://en.wikipedia.org/wiki/File_eXchange_Protocol
cleaver 15y ago
Does nobody use rsync? I see no mention in the comments here (and the original site is down). There's a few command line parameters to learn, but after that it is so simple, efficient and reliable that there is no need for a GUI client such as the various FTP FTP clients. I typically write scripts for specific purposes, syncing only the files that have changed. Using my SSH key means I don't have to type the password all the time.
moe 15y ago
Sign me up, but where's the alternative? WebDAV is a trainwreck. SFTP could be nice but the OpenSSH impl falls terribly short as a FTPd replacement (the most useful implementation is ironically the one in ProFTPd). Sendfile never went anywhere. Network filesystems don't cut the FTP use-case either. People don't use FTP because they like it. They use it for the lack of a viable alternative.
- peter_l_downs 15y ago
  cleaver and sehugg have both mentioned rsync, which I also think is a viable alternative. If possible, I try to use scp - is there any reason neither of these are viable alternatives? The only reason I ever use ftp is because I'm forced to with my godaddy hosting.
  - moe 15y ago
    For many use-cases rsync and scp are not adequate. For example neither can provide a file-listing or interactively walk a tree, which is essential for a wide range of push-based or fileserver-style applications.
    - jsight 15y ago
      rsync can provide file-listings: jsight@jsight-ubuntudesktop:~$ rsync localhost:/home/jsight/hackernews/ drwxrwxr-x 4096 2012/01/29 20:02:48 . -rw-rw-r-- 0 2012/01/29 20:02:36 tmp1 -rw-rw-r-- 0 2012/01/29 20:02:38 tmp2 -rw-rw-r-- 0 2012/01/29 20:02:39 tmp3 -rw-rw-r-- 11 2012/01/29 20:02:45 tmp4 -rw-rw-r-- 11 2012/01/29 20:02:46 tmp5 -rw-rw-r-- 11 2012/01/29 20:02:48 tmp6
      - moe 15y ago
        Thanks! That was new to me. Still, can it also serve as an interactive shell with mkdir, rename etc.? An rsync-based ftp-clone would indeed be my favorite for a replacement.
hackermom 15y ago
I think SFTP is an excellent alternative to FTP (as well as FTP over SSL/TLS). It's a whole lot safer than FTP and it solves the archaic annoyance of data ports. Sadly the OpenSSH solution is over the top awkward when it comes to setting up chrooted access, but it works. Introducing something entirely new at this point makes no sense - the situation isn't really as serious as some people want to make it out to be.
- antoncohen 15y ago
  > Sadly the OpenSSH solution is over the top awkward when it comes to setting up chrooted access, but it works. With OpenSSH it's pretty easy to setup users in a chroot, with no shell access. The hard part is if you want logging (to audit what they do), then you need to create a log device in their chroot, which adds complication, and you need OpenSSH 5.2+. sshd_config: Subsystem sftp internal-sftp -l VERBOSE Match Group sftponly ChrootDirectory %h ForceCommand internal-sftp -f AUTH -l VERBOSE Make user1 only be able to use SFTP: groupadd sftponly; usermod -a -G sftponly user1; usermod -s /sbin/nologin user1 If you actually want the "-l VERBOSE" logging to work you need to create ~user1/dev, then modify rsyslog.conf with: $AddUnixListenSocket /path/to/user1/home/dev/log
- sarnowski 15y ago
  Maybe you missed the OpenSSH 4.9 release? http://www.openssh.com/txt/release-4.9 http://www.openssh.com/txt/release-4.9 Since then, it is easy to setup a chroot'ed account which easily acts like an ftp server. Have a look at ChrootDirectory and the internal-sftp subsystem.
  - hackermom 15y ago
    No, in fact I update every 6 months ever since I started using OpenBSD back in 1999. The problem isn't setting up OpenSSH to offer chrooted sftp access. My gripe is that the sftp subsystem for no solid reason requires that a user directory is root-owned in order to chroot even when the user account experiences a forced sftp response (that is, denying shell access, making it an sftp-only account).
ew 15y ago
The reason FTP is still so pervasive is that FTP clients use a similar interface to the file browser on a computer. Your average person has no idea that the command line even exists. Yes, FTP isn't a very good protocol, but the interface is identical for SFTP.
jbarham 15y ago
Plan 9's 9P is a nice file protocol: http://plan9.bell-labs.com/sys/man/5/INDEX.html http://plan9.bell-labs.com/sys/man/5/INDEX.html. A client implementation has been available for Linux since version 2.6 (http://cm.bell-labs.com/wiki/plan9/v9fs/index.html http://cm.bell-labs.com/wiki/plan9/v9fs/index.html).
- lubutu 15y ago
  Unfortunately 9P2000 tends to perform badly over high latency connections. There was talk of writing a new version that would allow clients to group certain messages, but nothing ever came of it. Edit: I just found a very recent paper on Improving the performance of Styx based services over high latency links (Styx being the Inferno name for 9P): http://gsyc.es/tr-docs/RoSaC-2011-2.pdf http://gsyc.es/tr-docs/RoSaC-2011-2.pdf
  - jff 15y ago
    Ah, yes, that would be the work on Op, which IIRC can act as a sort of proxy, meaning you shouldn't have to re-write any clients or servers. There is also an experimental implementation of 9P "streams", which actually resemble a passive FTP transfer--a separate TCP connection is negotiated via the 9P connection and then used solely to transfer file data.
hemancuso 15y ago
We've got a writeup about how much fun it was implementing a FUSE based filesystem that transports with FTP [ExpanDrive] http://blog.expandrive.com/2009/02/02/ftp-considered-harmful/ http://blog.expandrive.com/2009/02/02/ftp-considered-harmful...
toyg 15y ago
So, what would it be a modern drop-in replacement for FTP? Requirements: * It should only deal with filesystem operations (i.e. SSH is way too problematic). * It should be firewall-, NAT-, proxy- and browser-friendly. * It should have cross-platform servers and clients, fully operational via CLI as well as GUI. * it should be secure (i.e. no http, thanks; https, maybe.) I'd say it's potential startup territory, except I cannot see any monetization opportunity. Also, the usual XKCD on standards: http://xkcd.com/927/ http://xkcd.com/927/
- soult 15y ago
  I don't see why you disqualify SSH here. If you want a system that only deals with filesystem operations, I suggest NFS. It is neither firewall-, NAT-, proxy- or browser-friendly, is not cross-platform (Windows support only if you pay extra) and is neither encrypted nor authenticated. But hey, it deals with filesystem operations only. If, on the other hand, you want a system that meets your last 3 requirements, just use SSH/SFTP.
  - toyg 15y ago
    SSH is a pain to secure properly, and the potential for mischief is huge. There is no sudo in FTP.
    - Hemospectrum 15y ago
      It's straightforward to permit SSH access but not shell logins.... if you can administer user and group settings. I guess stuff like scponly (http://freecode.com/projects/scponly http://freecode.com/projects/scponly) is useless if you're running Windows server-side.
    - kree10 15y ago
      Not that I'd like to encourage it, but there could be a sudo in FTP, along the lines of WU-FTPd's infamous support for a "SITE EXEC" command.
- jff 15y ago
  9P does a lot of these. It's just not very fast over high-latency links. On the other hand, it's extremely simple, can be easily encrypted, and does only filesystem operations. Oh, and there are clients and servers for all the major platforms and libraries for a lot of popular languages.
kenrik 15y ago
This should not be "FTP must die" but rather "someone needs to make a really nice GUI for rsync" I use rbsync to backup to S3 and it really works great but was a pain to setup as it's command line only.
sjwright 15y ago
Surely the best answer is for the HTTP 2.0 working group to require the updated protocol to include useful hooks for file-serving scenarios. Critically, there's only a few missing things in my estimation, and they revolve around resumable uploading and structured directory listings for GUI clients.
- pilif 15y ago
  WebDAV was invented as a HTTP based way for accessing remote files. In general, support on the clientside is very good (built-in clients are in every major OS). What hinders wide deployment is the server side: the most widely known implementation is mod_dav for apache and apache was never really made for a common use-case of FTP which is people using it with their unix account credentials for transmitting files. If you have access to an OSX server, have a look at all the hoops they had to jump through to allow WebDAV with mod_dav and still do that in the context of the corresponding system user. The other reason for FTP still being popular is legacy systems: over the years, I interfaced so many ERP systems for our product and usually, the only thing that customers can provide is good old FTP (or direct database access). As this scenario doesn't involve unix accounts, I would love to use WebDAV for all the reasons outlined in te article, but body supports it on their end, despite it being around for 20 yars or so.
joeyh 15y ago
It's perhaps worth noting that HTTP has essentially the same unspecified directory list problem as FTP. I've seen programs that screen scrape apache etc directory lists (as displayed when there is no index.html). Oddly, gopher was an intermediate protocol that got that right.
- dspillett 15y ago
  HTTP wasn't really intended as a file/directory transfer protocol the same though. The problems with FTP means that is fails at its primary function (unless you apply copious work-arounds). Many moons ago in my younger days I wrote an FTP based file/tree synchronisation tool. I have since vowed to never touch FTP again.
napierzaza 15y ago
Then mate write a hit piece on 56k modems
hcarvalhoalves 15y ago
Who still uses FTP? I believe SFTP has been the standard for quite some time already.
- RexRollman 15y ago
  The only thing I use FTP for these days is downloading free operating systems like OpenBSD and Arch Linux. For this kind of thing, I think FTP is fine.
SeoxyS 15y ago
FTP is dead. I haven't used the FTP protocol in years[1]… that's what we have SSH / SCP for. [1]: Before you say that my anecdotal evidence does not a fact make, my point is that there's nothing forcing us to use FTP today. Even the lousiest web host supports SFTP, and none of my machines or VPSs run an FTP server. There's no reason to proclaim that we need to kill FTP, because FTP is a non-issue in today's world. Sure, you can still use it, the same way you can still use telnet if you'd like. But practically nothing relies on it with no alternative.
jos3000 15y ago
I agree. There is no excuse for using FTP when we have brilliant alternatives like MegaUpload.
mjwalshe 15y ago
Awww didums if the OP can't understand and use a simple protocol like FTP good job he never had to work with grown up OSI standards - maybe they should stick to working at mc Donalds. And those mentioning rsync its not used for the same use cases as ftp. ftp is usefull for quickly transfereing a few files between systems - rsysnc is used for totaly diferent
jokull 15y ago
Anyone remember Hotline? That was a great protocol/client/server for file transfers.

174 points by werdnanoslen 15y ago

Text-only Google cache: http://webcache.googleusercontent.com/search?q=cache:6jeyroOZZnsJ:mywiki.wooledge.org/FtpMustDie&hl=en&gl=us&strip=1 http://webcache.googleusercontent.com/search?q=cache:6jeyroO...

dredmorbius 15y ago

Coral cache: http://mywiki.wooledge.org.nyud.net/FtpMustDie http://mywiki.wooledge.org.nyud.net/FtpMustDie

evmar 15y ago

You might be amused to browse Chrome's implementation of FTP, which has separate modules for each of the different types of FTP servers you might encounter, as each emits the "ls -l" output in a different way and that is (I guess) the only way to get file sizes. http://git.chromium.org/gitweb/?p=chromium.git;a=tree;f=net/ftp;h=a9ccad9efc957926f20f8825a4ffc95c24673729;hb=HEAD http://git.chromium.org/gitweb/?p=chromium.git;a=tree;f=net/... see all the "directory listing" files. The unit tests are full of scary cases like // Tests for "ls -l" style listing in Russian locale (note the swapped // parts order: the day of month is the first, before month).

toyg 15y ago
That's actually the standard European order. Testing the Russian locale might be a scary proposition, but certainly not for this specific reason.
- sjwright 15y ago
  It's not just the standard European order, it's the standard order for nearly everywhere on the planet except: > MDY: Belize, USA, parts of Canada, Philippines, Saudi Arabia > YMD: Japan, China, Iran, small bits of Europe > DMY: Probably 3/4 of the planet's land surface That said, YMD should make the most sense (and is most consistent with the universally accepted HMS time format). I try to use YMD wherever I can.
  - reitzensteinm 15y ago
    YMD is logical, because it puts the most significant number on the left. But both DMY and HMS are consistent in another way - the information is ordered from most to least important for every day access, from left to right. A vast majority of dates on advertising, tickets, timetables etc actually leave off the year, unless it's ambiguous. Similarly, one might say something like 'on the 4th', which implies 'the 4th of this month' (or the 4th of next month, if applicable). This is the reverse of HMS, where the most significant number (the hour) is most important, and seconds are basically ignored day to day. MDY, on the other hand, should be taken out the back and shot (IMO).
    - wvenable 15y ago
      > the information is ordered from most to least important for every day access I argue that MDY is ordered from most to least important for every day access. Year is easily implied so it is last so that leaves month first and then day. You'd be surprised how many Canadians, which is officially DMY, actually prefer MDY.
    - eslaught 15y ago
      In my opinion YMD is the most useful because it is the most visually distinct from the other two. A date like January 2nd 2012, when arranged in either MDY or DMY format, is difficult to distinguish from February 1st. YMD on the other hand is obvious, independent of what the reader is used to.
      - trebor 15y ago
        MDY is syntactically correct English: January 30th, 2012. Or in older form (where it would have come from): "Dated: January the 30th, in the year of our Lord 2012." Even the US Constitution is dated this way. I strongly prefer this way because it makes the most sense in the English language. For programmatic sorting YMD makes sense because is automatically sorts by Y, M, D, if compared as a string. (EG: on a filesystem.)
    - sunchild 15y ago
- dchest 15y ago
  The comment says "Russian" because they really test the month name in Russian: "23 май 2011". -rwxrwxr-x 1 ftp ftp 123 23 \xd0\xbc\xd0\xb0\xd0\xb9 2011 test The note about day, month order is useful because all other tests have "%b %e" format and rows are full of other numbers: drwxr-xr-x1732 266 111 90112 Jun 21 2001 .rda_2 http://git.chromium.org/gitweb/?p=chromium.git;a=blob;f=net/ftp/ftp_directory_listing_parser_ls_unittest.cc;h=b14d5c075a9fa6d65d58377139c7f7aadba5e9fa;hb=HEAD#l103 http://git.chromium.org/gitweb/?p=chromium.git;a=blob;f=net/...

TwoBit 15y ago

Time and time again I resort to FTP to transfer files between computers that won't talk to each other via system-provided file systems. Mac<->Windows file sharing is so unreliable that I don't even try to use the system any more and just go straight to FTP. But yeah I hate the protocol itself.

wollw 15y ago
Have you tried using SCP? I don't know if Windows comes with a client but a quick search turned up a few options. It is my goto method for transferring files these days but I admittedly don't use Windows.
- eCa 15y ago
  WinSCP has served me well over the years: http://www.winscp.net/ http://www.winscp.net/
- ecaron 15y ago
  I'd love to know the transfer speed differences between SCP, SFTP and FTP - because when I can view both ends of the wire then security is less of a concern than transfer time.
  - kronusaturn 15y ago
    The difference is negligible when both machines are fast enough to encrypt/decrypt at wire speed, otherwise FTP is faster.
  - astrodust 15y ago
    The overhead of encryption is marginal and the advantage of being able to compress the stream is considerable. scp and rsync, which is a fantastic companion, allows on-the-fly compression of content to get truly impossible speeds over the wire.
    - zoobert 15y ago
      It is marginal if you want to transfer a file from time to time but if you are in a business where you transfer thousands to millions of files everyday it makes a big difference. FTP is still used because its interface is simple and well-known: good old file-directory paradigm that everybody knows. It is also used because it runs on every platform. I am in favour of having a new protocol but it should: - be faster than FTP (would have to use UDP instead of TCP here). - Firewall friendly. For that, you could use one port to do everything and forget about the data connection (Is it feasible with UDP as you would need to control the flow ?) - Being FTP compliant from the interface point of view. You should be able to replace your ftp client with your fastTP protocol and everything would run. - Add optional strong security. Possibility to encrypt or not the data stream and encrypt the password negotiation. If we have this and it is marketed well then it could replace FTP.

omh 15y ago

FTP is definitely outdated, and there are better uses in many circumstances. But I find some of these criticisms a bit odd. The client listening was largely solved by 'passive' mode, and just about every server and client supports this now. The firewall and NAT interaction is awkward, but most modern firewalls can deal with this automatically (as long as there's no SSL involved) And yes, the RFC is 20 years old. But so are many RFCs for long-established veteran protocols that we use all over the net. In many cases I'd be happy to see FTP replaced (all those anonymous FTP servers may as well be HTTP now), but it's really not that bad.

tptacek 15y ago
PASV mode came with its own security drama, and doesn't address the core issue with the (absolutely pointless) multiple-connection model of FTP. And so, yes, firewalls and NATs "interact" with FTP --- because they all had to be hacked specifically to deal with the FTP protocol, which moots your rebuttal --- but in doing so they create additional security issues, like the NAT pinning stuff Samy Kamkar posted last year. It shouldn't be necessary for middleboxes to hack in support for protocols by in-place editing TCP streams and dynamically changing filter/translation rules based on stuff that happens inside of connections. But, thanks to FTP, they do have to do that. And for what? FTP is even on its own terms a terrible file transfer protocol! For instance, look at how it handles (or, commonly, doesn't handle) file listings. FTP is an anachronism. It has no more reason to exist today than TFTP --- both were designed to make up for constraints in client software that simply no longer exist anywhere.
- justincormack 15y ago
  Have used tftp for net booting, those legacy bioses cant do anything else, as it really is trivial...
  - tptacek 15y ago
    Has there been a piece of equipment shipped in the last 10 years that legitimately couldn't get a TCP running to bootstrap itself? That's why TFTP exists: to avoid the need for TCP.

charliesome 15y ago

> The LIST command is Maximum Bullshit. And I'm not even touching the subject of opening a data connection to send the list itself. Did you know that the specification doesn't tell anything about WHAT should be sent as the list? Well it's a good thing that just about every FTP server in existence supports the MLSD command then.

pygorex 15y ago

FTP is like religion: poorly designed at inception, implemented a thousand different ways and accepted as normal only because it's ubiquitous.

sjwright 15y ago
Except unlike religion, FTP began with a legitimate use-case scenario.
- rplnt 15y ago
  Don't know about all religions but some began with clear use-cases. To retain (or even achieve) power over people. For example The Old Testament was written/compiled for this very reason[1]. 1. http://books.google.com/books?id=iS7rQwAACAAJ http://books.google.com/books?id=iS7rQwAACAAJ
Nelson69 15y ago
FTP is kind of dying out though. Now the religion of SMTP.. I often wonder how the spam situation would look if we redid SMTP and required certified TLS. What's worse, we still use SMTP for real stuff every day, I get emails from Etrade, Fidelity and others that I simply don't want flying through cyberspace decrypted even if they don't contain real information.
gaius 15y ago
The people who designed FTP were not stupid: they did the best they could operating under constraints that kids who have grown up with all the serious work of implementing the Internet already done for them cannot imagine. It's easy, with decades of hindsight, to claim you could do better. I bet you couldn't.

thristian 15y ago

It's been a long time since I've been involved with transferring files to and from clients, but even five years or so ago I recall enabling IIS's WebDAV mode in preference to handling customer-support calls about getting FTP working. Of course, WebDAV is still a ball of horror, even if it's not as bad as FTP, so please let's all stick to SFTP from now on?

eli 15y ago
So you had XP boxes connecting via WebDAV? I think I'd take FTP over having to support that.
- thristian 15y ago
  Indeed. It worked pretty well, actually - in the standard Windows Explorer 'mount network drive' dialog, you could type an HTTP URL and get an ordinary-looking Explorer window and drag files to and fro.

alecco 15y ago

Browser HTTP file upload is still terrible for file transfer. One of the last bastions of Flash, regretfully.

tptacek 15y ago
Browsers are terrible at file upload. The weakness here isn't really in the protocol.
- alecco 15y ago
  That's why I said browsers. But only recently with HTML5 XHR file uploads we could have progress and status reported in-page (instead of poorly in the status bar, if any). And it's still not good enough and definitely not widely available.
  - tptacek 15y ago
    But the choice isn't between "browsers doing FTP" and "browsers doing HTTP"; it's between special-purpose FTP software and (somewhat hypothetical, if you exclude Transmit) special-purpose HTTP software.
  - axiak 15y ago
    Huh? As long as you have XHR and JS, you can do file upload progress bars. Gmail has been doing it for years, and I've certainly had implementations going back to 2007. Now, if the upload is failed for whatever reason (i.e. the file is rejected in-progress), then the upload failure mode is terrible unless you resort to the usual iframe hackery (Connection Reset screen). HTML5 XHR file upload is certainly a cleaner solution and something everybody has wanted, but it's certainly not needed for progress and status reported in-page without flash.
    - alecco 15y ago
      OK. Small nitpick, Gmail used a Flash uploader for many years, at least until 2009: http://ajaxian.com/archives/multi-file-upload-in-the-flickr-and-gmail-house http://ajaxian.com/archives/multi-file-upload-in-the-flickr-...

sehugg 15y ago

I don't think FTP must die so much as RSYNC MUST LIVE. (Seriously, people forget how awesome rsync is. And it tunnels/compresses nicely over SSH)

kree10 15y ago
Also don't forget that rsync does not overwrite a remote file until it's completely uploaded. Compare to ftp, sftp and scp, which overwrite the file from the start, possibly breaking your site during the transfer (or definitely breaking your site if the transfer dies part-way through).
- plq 15y ago
  > Also don't forget that rsync does not overwrite a remote file until it's completely uploaded. which is really troublesome when you're transferring a big file and the target doesn't have enough space for a second copy of that particular file. I, for one, would prefer that "feature" to be optional.
  - kree10 15y ago
    Take a look at rsync's "--inplace" option. I think this does what you want.

nviennot 15y ago

There is something pretty cool that one can do with the FTP protocol: FXP transfers to perform an inter-server file transfer: https://en.wikipedia.org/wiki/File_eXchange_Protocol https://en.wikipedia.org/wiki/File_eXchange_Protocol

cleaver 15y ago

Does nobody use rsync? I see no mention in the comments here (and the original site is down). There's a few command line parameters to learn, but after that it is so simple, efficient and reliable that there is no need for a GUI client such as the various FTP FTP clients. I typically write scripts for specific purposes, syncing only the files that have changed. Using my SSH key means I don't have to type the password all the time.

moe 15y ago

Sign me up, but where's the alternative? WebDAV is a trainwreck. SFTP could be nice but the OpenSSH impl falls terribly short as a FTPd replacement (the most useful implementation is ironically the one in ProFTPd). Sendfile never went anywhere. Network filesystems don't cut the FTP use-case either. People don't use FTP because they like it. They use it for the lack of a viable alternative.

peter_l_downs 15y ago
cleaver and sehugg have both mentioned rsync, which I also think is a viable alternative. If possible, I try to use scp - is there any reason neither of these are viable alternatives? The only reason I ever use ftp is because I'm forced to with my godaddy hosting.
- moe 15y ago
  For many use-cases rsync and scp are not adequate. For example neither can provide a file-listing or interactively walk a tree, which is essential for a wide range of push-based or fileserver-style applications.
  - jsight 15y ago
    rsync can provide file-listings: jsight@jsight-ubuntudesktop:~$ rsync localhost:/home/jsight/hackernews/ drwxrwxr-x 4096 2012/01/29 20:02:48 . -rw-rw-r-- 0 2012/01/29 20:02:36 tmp1 -rw-rw-r-- 0 2012/01/29 20:02:38 tmp2 -rw-rw-r-- 0 2012/01/29 20:02:39 tmp3 -rw-rw-r-- 11 2012/01/29 20:02:45 tmp4 -rw-rw-r-- 11 2012/01/29 20:02:46 tmp5 -rw-rw-r-- 11 2012/01/29 20:02:48 tmp6
    - moe 15y ago
      Thanks! That was new to me. Still, can it also serve as an interactive shell with mkdir, rename etc.? An rsync-based ftp-clone would indeed be my favorite for a replacement.

hackermom 15y ago

I think SFTP is an excellent alternative to FTP (as well as FTP over SSL/TLS). It's a whole lot safer than FTP and it solves the archaic annoyance of data ports. Sadly the OpenSSH solution is over the top awkward when it comes to setting up chrooted access, but it works. Introducing something entirely new at this point makes no sense - the situation isn't really as serious as some people want to make it out to be.

antoncohen 15y ago
> Sadly the OpenSSH solution is over the top awkward when it comes to setting up chrooted access, but it works. With OpenSSH it's pretty easy to setup users in a chroot, with no shell access. The hard part is if you want logging (to audit what they do), then you need to create a log device in their chroot, which adds complication, and you need OpenSSH 5.2+. sshd_config: Subsystem sftp internal-sftp -l VERBOSE Match Group sftponly ChrootDirectory %h ForceCommand internal-sftp -f AUTH -l VERBOSE Make user1 only be able to use SFTP: groupadd sftponly; usermod -a -G sftponly user1; usermod -s /sbin/nologin user1 If you actually want the "-l VERBOSE" logging to work you need to create ~user1/dev, then modify rsyslog.conf with: $AddUnixListenSocket /path/to/user1/home/dev/log
sarnowski 15y ago
Maybe you missed the OpenSSH 4.9 release? http://www.openssh.com/txt/release-4.9 http://www.openssh.com/txt/release-4.9 Since then, it is easy to setup a chroot'ed account which easily acts like an ftp server. Have a look at ChrootDirectory and the internal-sftp subsystem.
- hackermom 15y ago
  No, in fact I update every 6 months ever since I started using OpenBSD back in 1999. The problem isn't setting up OpenSSH to offer chrooted sftp access. My gripe is that the sftp subsystem for no solid reason requires that a user directory is root-owned in order to chroot even when the user account experiences a forced sftp response (that is, denying shell access, making it an sftp-only account).

ew 15y ago

The reason FTP is still so pervasive is that FTP clients use a similar interface to the file browser on a computer. Your average person has no idea that the command line even exists. Yes, FTP isn't a very good protocol, but the interface is identical for SFTP.

jbarham 15y ago

Plan 9's 9P is a nice file protocol: http://plan9.bell-labs.com/sys/man/5/INDEX.html http://plan9.bell-labs.com/sys/man/5/INDEX.html. A client implementation has been available for Linux since version 2.6 (http://cm.bell-labs.com/wiki/plan9/v9fs/index.html http://cm.bell-labs.com/wiki/plan9/v9fs/index.html).

lubutu 15y ago
Unfortunately 9P2000 tends to perform badly over high latency connections. There was talk of writing a new version that would allow clients to group certain messages, but nothing ever came of it. Edit: I just found a very recent paper on Improving the performance of Styx based services over high latency links (Styx being the Inferno name for 9P): http://gsyc.es/tr-docs/RoSaC-2011-2.pdf http://gsyc.es/tr-docs/RoSaC-2011-2.pdf
- jff 15y ago
  Ah, yes, that would be the work on Op, which IIRC can act as a sort of proxy, meaning you shouldn't have to re-write any clients or servers. There is also an experimental implementation of 9P "streams", which actually resemble a passive FTP transfer--a separate TCP connection is negotiated via the 9P connection and then used solely to transfer file data.

hemancuso 15y ago

We've got a writeup about how much fun it was implementing a FUSE based filesystem that transports with FTP [ExpanDrive] http://blog.expandrive.com/2009/02/02/ftp-considered-harmful/ http://blog.expandrive.com/2009/02/02/ftp-considered-harmful...

toyg 15y ago

So, what would it be a modern drop-in replacement for FTP? Requirements: * It should only deal with filesystem operations (i.e. SSH is way too problematic). * It should be firewall-, NAT-, proxy- and browser-friendly. * It should have cross-platform servers and clients, fully operational via CLI as well as GUI. * it should be secure (i.e. no http, thanks; https, maybe.) I'd say it's potential startup territory, except I cannot see any monetization opportunity. Also, the usual XKCD on standards: http://xkcd.com/927/ http://xkcd.com/927/

soult 15y ago
I don't see why you disqualify SSH here. If you want a system that only deals with filesystem operations, I suggest NFS. It is neither firewall-, NAT-, proxy- or browser-friendly, is not cross-platform (Windows support only if you pay extra) and is neither encrypted nor authenticated. But hey, it deals with filesystem operations only. If, on the other hand, you want a system that meets your last 3 requirements, just use SSH/SFTP.
- toyg 15y ago
  SSH is a pain to secure properly, and the potential for mischief is huge. There is no sudo in FTP.
  - Hemospectrum 15y ago
    It's straightforward to permit SSH access but not shell logins.... if you can administer user and group settings. I guess stuff like scponly (http://freecode.com/projects/scponly http://freecode.com/projects/scponly) is useless if you're running Windows server-side.
  - kree10 15y ago
    Not that I'd like to encourage it, but there could be a sudo in FTP, along the lines of WU-FTPd's infamous support for a "SITE EXEC" command.
jff 15y ago
9P does a lot of these. It's just not very fast over high-latency links. On the other hand, it's extremely simple, can be easily encrypted, and does only filesystem operations. Oh, and there are clients and servers for all the major platforms and libraries for a lot of popular languages.

kenrik 15y ago

This should not be "FTP must die" but rather "someone needs to make a really nice GUI for rsync" I use rbsync to backup to S3 and it really works great but was a pain to setup as it's command line only.

sjwright 15y ago

Surely the best answer is for the HTTP 2.0 working group to require the updated protocol to include useful hooks for file-serving scenarios. Critically, there's only a few missing things in my estimation, and they revolve around resumable uploading and structured directory listings for GUI clients.

pilif 15y ago
WebDAV was invented as a HTTP based way for accessing remote files. In general, support on the clientside is very good (built-in clients are in every major OS). What hinders wide deployment is the server side: the most widely known implementation is mod_dav for apache and apache was never really made for a common use-case of FTP which is people using it with their unix account credentials for transmitting files. If you have access to an OSX server, have a look at all the hoops they had to jump through to allow WebDAV with mod_dav and still do that in the context of the corresponding system user. The other reason for FTP still being popular is legacy systems: over the years, I interfaced so many ERP systems for our product and usually, the only thing that customers can provide is good old FTP (or direct database access). As this scenario doesn't involve unix accounts, I would love to use WebDAV for all the reasons outlined in te article, but body supports it on their end, despite it being around for 20 yars or so.

joeyh 15y ago

It's perhaps worth noting that HTTP has essentially the same unspecified directory list problem as FTP. I've seen programs that screen scrape apache etc directory lists (as displayed when there is no index.html). Oddly, gopher was an intermediate protocol that got that right.

dspillett 15y ago
HTTP wasn't really intended as a file/directory transfer protocol the same though. The problems with FTP means that is fails at its primary function (unless you apply copious work-arounds). Many moons ago in my younger days I wrote an FTP based file/tree synchronisation tool. I have since vowed to never touch FTP again.

napierzaza 15y ago

Then mate write a hit piece on 56k modems

hcarvalhoalves 15y ago

Who still uses FTP? I believe SFTP has been the standard for quite some time already.

RexRollman 15y ago
The only thing I use FTP for these days is downloading free operating systems like OpenBSD and Arch Linux. For this kind of thing, I think FTP is fine.

SeoxyS 15y ago

FTP is dead. I haven't used the FTP protocol in years[1]… that's what we have SSH / SCP for. [1]: Before you say that my anecdotal evidence does not a fact make, my point is that there's nothing forcing us to use FTP today. Even the lousiest web host supports SFTP, and none of my machines or VPSs run an FTP server. There's no reason to proclaim that we need to kill FTP, because FTP is a non-issue in today's world. Sure, you can still use it, the same way you can still use telnet if you'd like. But practically nothing relies on it with no alternative.

jos3000 15y ago

I agree. There is no excuse for using FTP when we have brilliant alternatives like MegaUpload.

mjwalshe 15y ago

Awww didums if the OP can't understand and use a simple protocol like FTP good job he never had to work with grown up OSI standards - maybe they should stick to working at mc Donalds. And those mentioning rsync its not used for the same use cases as ftp. ftp is usefull for quickly transfereing a few files between systems - rsysnc is used for totaly diferent

jokull 15y ago

Anyone remember Hotline? That was a great protocol/client/server for file transfers.