Software that treats a folder as the input to a queue?

Question

I would like to receive files by FTP in a folder, and then put each file on a queue so that different workers can process it.however my googling attempts haven't return satisfactory results?Maybe I'm not searching for the right keywords? How do people with legacy systems, that receive text files through FTP, process each file through a pipeline of work in a reliable way?

wmf · Accepted Answer

Bunch of suggestions here https://stackoverflow.com/questions/4060212/how-to-run-a-she... https://superuser.com/questions/181517/how-to-execute-a-comm...
Also apparently systemd can do something like this. "DirectoryNotEmpty= may be used to watch a directory and activate the configured unit whenever it contains at least one file." https://www.freedesktop.org/software/systemd/man/systemd.pat...
The keyword is inotify since that's the (Linux) API that allows watching files/directories.

mtmail · Answer

We had to build our own 10 years ago. One thing we learned is to never allow FTP users to change, overwrite or delete files once uploaded. There are feature flags for that, I think we were using pureftp. Then our script moved all files out of the directory before processing.