On 17 February 2017 at 15:41, Stewart C. Russell via talk <talk@gtalug.org> wrote:
I need to watch a folder on a network share (a scanner) and see when new files are created. There are a couple of special things about this location:
* It's over CIFS. There's nothing I can do about that. I think that means I can't use Inotify.
* It doesn't have an accurate clock for timestamping. There's nothing I can do about that, either.
* If the share is being written to by the device, it effectively disappears: stat() complains loudly.
* It reuses the lowest available file name of the form “EPSON%03d.(PDF|JPG)”, so synching the share to a sensible filesystem might overwrite files accidentally.
* I don't need to watch it very often; a few times per hour would be fine.
I tried using Perl's File::ChangeNotify::Watcher, which is supposed to pick the smartest method available. Unfortunately, this seems to be calling stat() every two seconds, resulting in:
1. Messages (one per existing file on the share) saying that the scanner has deleted them. This happens when someone starts scanning
2. An error message from stat() every two seconds while the device is scanning
3. Messages saying that all of the old files (plus the new one) have been created.
I was naïvely hoping for a mechanism that would report “Hey, you have a new file called $filename” but that seems to have been a pipedream. What do People Who Actually Know What They're Doing use, please?
How's your coding? I'm visualizing this in Bash on a cron job. This is purely theory, and I apologize if it's not useful. It seems like it would work if I've understood the circumstances correctly ... - every fifteen minutes get a checksum on all the remote files - get checksums on the files in the local copy of the folder - if a checksum exists on the remote but not the local ('uniq -c ...' may help), copy the file and datestamp it (ie. 'cp -vi //CIFS/a.tif /localcopy/a.2017-02-17.1610.tif'). - hmm - might work to embed the checksum in the local filename for ease of access ... There's a bit of juggling to be done on comparing checksums, but it shouldn't be too hard. Would this work? -- Giles http://www.gilesorr.com/ gilesorr@gmail.com