Yes, this is indeed a problem! There need to be consistency checks before uploading!
First: Serious corruption will -- afaics -- occur, when you use Rclone while there is an ongoing commit/backup by one of your computers. You said that the laptops backup irregular and afaics there is no checking for this involved here.
You hence need at least to add the `pidof -x borg` check from your backup.sh to your Rclone script.
Second: Mild corruption will occur when the backup process breaks at some point. E.g. laptop shuts off/powers off, WiFi breaks, ... You said that you want things to work unnoticed in the background. That makes a realistic chance for this to happen. Divide the risk by the awareness of the end user. :-)
Usually borg is able to repair these cases. BUT i) You have to notice and do this manually ii) I don't know whether this always works/this is bullet proof iii) In the current setup described in the post, there is no checking if backup succeeded. Using CRON in the background, it might happen that your backup fails once and all subsequent backups fail unnoticed.
Third: There are myriads of other potential vectors (e.g. data degradation [1], failing hardware [2], ...). You might add a `borg check` as well as a separate checksum comparison of both archives prior to Rclone.
borg requires you to choose an encryption mode on creation of the repository. Hence you really need to want your files to lie unencrypted somewhere to do so. Otherwise BLAKE2b and SHA-256 are pretty solid and can be considered safe for uploading to somewhere. And if you're really paranoid or eager, you can set your ssh encryption accordingly to not use a weaker one for uploading.
Authored Comments
Yes, this is indeed a problem! There need to be consistency checks before uploading!
First: Serious corruption will -- afaics -- occur, when you use Rclone while there is an ongoing commit/backup by one of your computers. You said that the laptops backup irregular and afaics there is no checking for this involved here.
You hence need at least to add the `pidof -x borg` check from your backup.sh to your Rclone script.
Second: Mild corruption will occur when the backup process breaks at some point. E.g. laptop shuts off/powers off, WiFi breaks, ... You said that you want things to work unnoticed in the background. That makes a realistic chance for this to happen. Divide the risk by the awareness of the end user. :-)
Usually borg is able to repair these cases. BUT i) You have to notice and do this manually ii) I don't know whether this always works/this is bullet proof iii) In the current setup described in the post, there is no checking if backup succeeded. Using CRON in the background, it might happen that your backup fails once and all subsequent backups fail unnoticed.
Third: There are myriads of other potential vectors (e.g. data degradation [1], failing hardware [2], ...). You might add a `borg check` as well as a separate checksum comparison of both archives prior to Rclone.
Anyway, thank you for the nice article! :-)
[1] https://en.wikipedia.org/wiki/Data_degradation
[2] https://github.com/borgbackup/borg/issues/3602
borg requires you to choose an encryption mode on creation of the repository. Hence you really need to want your files to lie unencrypted somewhere to do so. Otherwise BLAKE2b and SHA-256 are pretty solid and can be considered safe for uploading to somewhere. And if you're really paranoid or eager, you can set your ssh encryption accordingly to not use a weaker one for uploading.