
Ask HN: Software that treats a folder as the input to a queue? - elviejo
I would like to receive files by FTP in a folder,
and then put each file on a queue so that different workers can process it.<p>however my googling attempts haven&#x27;t return satisfactory results?<p>Maybe I&#x27;m not searching for the right keywords?
How do people with legacy systems, that receive text files through FTP, process each file through a pipeline of work in a reliable way?
======
wmf
Bunch of suggestions here [https://stackoverflow.com/questions/4060212/how-to-
run-a-she...](https://stackoverflow.com/questions/4060212/how-to-run-a-shell-
script-when-a-file-or-directory-changes)
[https://superuser.com/questions/181517/how-to-execute-a-
comm...](https://superuser.com/questions/181517/how-to-execute-a-command-
whenever-a-file-changes)

Also apparently systemd can do something like this. "DirectoryNotEmpty= may be
used to watch a directory and activate the configured unit whenever it
contains at least one file."
[https://www.freedesktop.org/software/systemd/man/systemd.pat...](https://www.freedesktop.org/software/systemd/man/systemd.path.html)

The keyword is inotify since that's the (Linux) API that allows watching
files/directories.

------
mtmail
We had to build our own 10 years ago. One thing we learned is to never allow
FTP users to change, overwrite or delete files once uploaded. There are
feature flags for that, I think we were using pureftp. Then our script moved
all files out of the directory before processing.

