I’m trying to come up with a elegant way of backing up my docker volumes. I don’t really care about my host and the data on the host, because everything I do happens inside my docker containers and the mapped volumes. Some containers use mapped paths, but some others use straight up docker volumes.
I’ve started writing a script that inspects the containers, reads all the mount paths and then spins up another container to tar all the mounted paths.
docker run --rm --volumes-from $container_name-v /data/backup:/backup busybox tar cvf /backup/$container_name.tar $paths
So far so good, this works and I can write all backups to my storage and later sync them to an offsite backup space.
But error handling, (nice)logging and notifications using ntfy in case of success / errors / problems is going to suck in a bash script. Local backup file rollover and log file rollover also just suck if I have to do all this by hand. I’m able to use other languages to write this backup util but I don’t want to start this project if there already is a ready made solution.
So the question, is there a utility that can simply schedule arbitrary bits of script, write nicer logs for these script bits, do file rollovers and run another script on success / error?
All the backup programs that I can find are more focused on backing up directories, permissions and so on.
This is a classic conundrum.
Containers and their volumes are supposed to be ephemeral (right?).
Yet we use them to run little apps where we configure settings etc in the app which we would like to “keep” - thus back up. Yes in a proper set up you would hook your container up to something that is not ephemeral like a database somewhere, but often we just want an app, see it’s got a self contained docker image, and just run it.
Whilst not in the spirit of things… I’ve tried using Borg backup however it just fails due to random permissions on the volumes.
I should spend more time looking into it but haven’t the time right now, could be the solution is specific to the app/container but the simplicity of just backing up a /volumes/* directory is soooo tempting…
Edit upon reflection, what about a sudo cron tab to zip volumes and set useful permissions on the zip. Then Borg to backup the zip. Borg (or at least vorta) can easily run scripts before/after and pass variables relating to the backup though.
I can run the scripts by hand but I have to do everything else (logging, rollovers, notifications, etc.) by hand too, that is what I wanted to avoid.
Permissions are not a big deal inside the containers and even If there is an issue, this is for my home nas. I don’t really care If I have to spend a couple of hours to fix broken permissions after restoring something from a backup.