Plans should be able to backup to multiple repos

[MrModest]

First of all, thank you for the backrest. Even though it might be better, it's already cool enough because it has at least a workaround for having a same backup in a multiply places.

Is your feature request related to a problem? Please describe.
Right now, I need to duplicate repository and plans per each "actual repo" per mirror.

For example, I want to have one repo for docker volumes data, another for immich photos and another for paperless docs. I also want each of them to be stored in my local machine, in my cloud drive and let's say S3 bucket. So, I have 3 "actual repos" and 3 destinations to store backups. Which means I have to create 3×3 = 9 repos and also 9 plans. Which is a huge boilerplate.

Describe the solution you'd like
Instead of creating an additional repo with essentially the same config (with only different repo URI), it would be nice to have an ability to add/remove mirrors to the repo.
Like, in the described case, I would have 3 repos and for each of them I would add a couple of mirrors.

And the backrest under the hood will just run a job with rclone copying the backup folder from the source repo to the mirrors' destination.

Also, instead of having an additional plan per mirror, it would be great to be able to choose which mirrors to use for this plan. So, after restic will finish backup to the source repo, backrest will start a rclone copy job to chosen mirrors.

Additional context
So, it's essentially what many people (I bet) are already doing, but with clutter scripting, and it would be nice to be able to control this via a nice UI :)

[inode64]

It would be something similar to tikect #465, allowing copying between repos

[garethgeorge]

Right — I’m hesitant to bake this into backrest based on some of the points on that thread (thanks for linking it @inode64!).

I think that using rclone with a script is a good workaround and reasonable if one understands / thinks through the possible pitfalls, but the footguns need thinking through if looking at built in support.

Specifically with rclone, I’m also somewhat hesitant to bake in support for automating binaries other than restic. A high quality script might be a good candidate for the cookbooks though.

[MrModest]

I see it as a feature of backrest, because it would be more convenient to just have an additional config and be able to monitor statuses and logs in the backrest along with other backup history.

I can configure a cron job that will run 'rclone sync' for me. But then the whole backup flow will be de-orginized and messy :(

Even though, everyone says that "3-2-1 is a must", there's no convenient all-in-one solution that will help you to do such backups. Always some hucky ways.

So, I hope, maybe 'backrest' can become this kind of tool eventually 😅

[garethgeorge]

To add a bit more context, I think a solution is needed for the fundamental pain point which is sending data to multiple repos, but I think the restic way to do this is running the backup operation multiple times.

This probably looks like allowing repos to inherit some settings from each other I.e. shared lists of paths and excludes.

Best practice in my mind is frequently to fast and cheap storage (read: on site), and less frequent scans and uploads to offsite storage which mitigates the cost of doing multiple scans of the filesystem.

[MrModest]

Seems like if to use autorestic instead of restic, the copying can be implemented easier: https://autorestic.vercel.app/location/options/copy

[bytebone]

This request is also of interest to me, but my situation seems to be a little different. Opposed to some other people asking for this, I am maintaining one giga-repo for all data (to maximize deduplication). There are currently 4 plans feeding into this repo, with huge differences in total size share. I now want to add a second repo to some of the 4 plans, those with the most important data and a smaller footprint.

Hence, it would be great if I were able to add multiple repositories to a plan. I neither mind nor really care if backrest runs the two backup jobs in sequence or parallel. In my head, this setting would run the same restic command for each defined repo, simply replacing the adjusting repo respectively. All other plan settings can (and should) be shared.

The multiple backups would then all appear in the same plan, and repo syncs should also query all configured repos in parallel.

For now, I'll see if I can make do with snapshot copying. But having this tightly integrated into the UI would be grand. Thank you!

[nicktheone]

Any new on this one? I'd like to be able to backup one plan to different repos and not just copy the repo to a different device/media because in case of file corruption I'd just copy around an unrecoverable backup. Doing multiple, separate backups to different repos (one in the cloud, one local in my case) sounds way more reliable and safe.

[timetheoretical]

Echoing @nicktheone, this restic forum discussion makes the case for having independent backups over copies, as random bitflips during a backup could propagate in a copy, but is less likely to occur in two separate backups.

As @garethgeorge proposes, I think a clean solution could be to let plans inherit options from a "parent" plan, like resticprofile's inheritance feature. That would be more general, and possible less complex to implement, than setting multiple mirrors for a specific plan.

[madeupandprobablydoesnotexist]

Echoing @nicktheone, this restic forum discussion makes the case for having independent backups over copies, as random bitflips during a backup could propagate in a copy, but is less likely to occur in two separate backups.

As @garethgeorge proposes, I think a clean solution could be to let plans inherit options from a "parent" plan, like resticprofile's inheritance feature. That would be more general, and possible less complex to implement, than setting multiple mirrors for a specific plan.

Wow that's really cool! Hope to see profiling soon, very useful.

[cbhushan]

As @garethgeorge proposes, I think a clean solution could be to let plans inherit options from a "parent" plan, like resticprofile's inheritance feature. That would be more general, and possible less complex to implement, than setting multiple mirrors for a specific plan.

I have somewhat similar, but simpler suggestion: Would it be possible to make it easier to create a new plan that starts from an existing plan except for repo-url & repo-password? May be "Add Plan" has an option like 'inherit config from ' & may be users can select which keys should be inherited. Note that this is suggesting to inherit configuration only at the time when a new plan is created and after that the new plan is completely independent of previous plan. I am guessing this would much simpler to implement & test than resticprofile's more complete inheritance feature.

My use case is that I have a few plans already configured for one repo and I want to basically add the same plan with few new destination repos.