Ignoring files or directories, or specifying files to be scanned

[pallinger]

I know this is quite a big feature, but could be nice to have: I have a lot of small backup files in a directory, and bitrot is progressing really slowly (0.1% in ~20 hours).
One simple implementation would be for bitrot to accept a predefined file-list, which could be generated by find or similar, that has already exclude options. I may start to work on it, if I have the time. :)

[ambv]

Wow, 0.1% in 20 hours is pretty awful. As for a predefined file list, I'm thinking about an ignore list instead. If you really want to track known files, you can easily do it with md5sum and friends.

[pallinger]

I do not mean tracking known files, I thought about being able to use bitrot as part of a toolchain. E.g.
find -type f -size +1M | grep -v '~$' | bitrot --file-list-from-stdin. So I did not think about using a static file list, but an on-the-fly-generated one.
Of course, a --exclude-file-list-from-stdin might also do the same trick.

[benshep]

I made a fork of the project to do this: https://github.com/benshep/bitrot

You can use --file-list (accepts a file or stdin) and --exclude-list (accepts files only, including wildcards).

[huyz]

@benshep Are you planning to update your fork for bitrot 1.0.0? (I need to exclude some Maildir directories that contain files that get frequently modified by IMAP syncing.)

[benshep]

I think it should be fairly straightforward to make a pull request from my fork. I'll try to find some time for that this week.

[p1r473]

@huyz My fork has been updated to 1.0 now
Took me all day!
I'd love some help unit testing, if anyone is able to help me test.

@pallinger My fork has both inclusion and exclusion lists

[benshep]

Great! I guess we can use your fork @p1r473 - thanks. That saves me a bunch of effort, and I'm certain you'll have done it better than I would ☺

[ambv]

@p1r473, wow, you updated the fork to 1.0? That's amazing. If you'd like to submit your support for ignoring files as a pull request, I'll accept it.

[huyz]

@p1r473 Great! I can't find the usage instructions for how to include and exclude. How does one do it?

[p1r473]

@p1r473, wow, you updated the fork to 1.0? That's amazing. If you'd like to submit your support for ignoring files as a pull request, I'll accept it.

There are just so many changes from my fork... I think I've more than doubled the line count

[p1r473]

@p1r473 Great! I can't find the usage instructions for how to include and exclude. How does one do it?

py bitrot.py --exclude-list exclude.txt --include-list include.txt
Include takes precedence over exclude

[huyz]

There are just so many changes from my fork... I think I've more than doubled the line count

@p1r473 You have made a lot of changes. Can you describe all the changes you've made? (Maybe add them to your version of the README?)

[p1r473]

There are just so many changes from my fork... I think I've more than doubled the line count

@p1r473 You have made a lot of changes. Can you describe all the changes you've made? (Maybe add them to your version of the README?)

They are already in my readme

[huyz]

@p1r473 You have made a lot of changes. Can you describe all the changes you've made? (Maybe add them to your version of the README?)

They are already in my readme

I meant, as a separate section so that it’s clear what features your fork adds to the original repo