In a previous article we wrote about some of the tools and services that we use on a regular basis to backup our Linux servers. Since then, Borg has become the tool that we rely on for most of our daily backup needs. It performs very well, it has all the features that we need, and it has proven to be very reliable and easy to maintain.
However, as a matter of principle, we don’t want to rely completely on just one backup tool. Ideally, we would like to have two completely separate backup systems in place, each maintaining backup sets at different offsite locations. We have been trialing Resticas that secondary backup system for the past few weeks.
Comparison
Borg and Restic are both deduplicating backup programs. When you backup a folder with either of these, each file in the folder is split into small chunks that are stored in a backup respository along with a unique identifier that depends on the contents of the chunk. If several files contain the same chunk, only a single copy of that chunk is stored in the repository. Restoring a file is then a matter of retrieving the individual chunks that made up the original file from the repository and writing them to disk.
The next time the same backup job is run, only chunks that have not been seen previously need be transferred and saved. That means that backups made after the initial run typically complete quickly, because the backup system just needs to identify any changes, transfer missing chunks to the repository, and create a new entry with information about this particular backup (an archive in Borg terminology or snapshot in Restic). In other words, backups made with Borg or Restic are incremental in nature, but each archive/snapshot can be restored as if it were a standalone, full backup.
Borg and Restic repositories can be stored as ordinary foldes and files, and they can be copied and moved around using standard file system utilities such as cp, mv or rsync. However, the files that are stored inside a repository can only be accessed using the program that was used to create the repository. I.e. if a backup was made with Borg, it can only be restored using Borg, and Restic backups must be restored using Restic.
Borg and Restic are also similar in that backups are encrypted (optionally, in the case of Borg). Both tools are designed to run on the machine being backed up, with full access to the local file system. Respositories can be local or remote, but neither tools can work in a “pull” configuration where backups are being initiated from a central backup server.
Borg and Restic are command-line only, however both programs support mounting a backup repository using fuse, which makes restoring individual files and folders somewhat easier compared to specifying them on the command line. Both tools have commands that can be used to verify the consistency of a backup repository, including options to verify that individual files can be correctly restored.
Finally, both are open source projects released under BSD licenses. Both projects are regularly updated and new releases issued.
Restic Advantages over Borg
Borg and Restic are similar enough that they can be considered competing alternatives, however each has some distinct features that can make it more attractive for certain use cases.
Borg is designed to work with repositories that are stored on the local file system and with remote repositories that can be accessed using SSH. In the remote repository case, Borg is expected to be installed on both the machine being backed up and on the machine hosting the repository files.
Restic, on the other hand, supports a number of storage backends for repositories:
- Local directory
- sftp
- REST server
- AWS S3 (either from Amazon or using the Minio server)
- OpenStack Swift
- BackBlaze B2
- Microsoft Azure Blob Storage
- Google Cloud Storage
Another difference between the two programs is related to deduplication. Borg is designed with the assumption that each machine being backed up will use its own repository. Letting multiple machines backup to the same repository can impact performance, and simultaneous backups from different machines to the same respository are not supported.
With Restic there is no such limitation – several machines can backup to the same repository without problems. That means that deduplication will not only take into consideration identitial chunks from files on a single machine, but from any of the machines that share a repository. In other words, if the same file exists on several computers, only one copy of that file will be stored in the repository. The same applies if individual chuncks from a large file that is found on several machines are identical – only one of those chunks is stored in the repository. In some cases this can lead to significant storage space savings.
Restic stores the key used to encrypt a repository inside the repository itself, and that key is then protected using an unlock key. Restic lets you create multiple unlock keys to a repository, so different machines sharing a repository can access it using different keys. However, different keys do not imply different access privileges – any of these keys can be used to access the entire repository. But being able to use different keys makes it easy to remove an individual machine from the repository if necessary, without having to change keys on all the other machines using the repository.
Borg Advantages over Restic
Generally speaking, Borg has a much larger feature set than Restic. Encryption is optional, which is nice if the medium that the repository is stored on is already encrypted at the file system level. You also have the option of encrypting without storing the encryption key in the repository, or you can store the key in the repository and unlock it using a passphrase (similar to Restic unlock keys).
Restic has no support at all for compression, but Borg lets you choose between no compression, lz4, zlib, and lzma. If you decide to compress using zlib or lzma you can also specify the compression level on a scale 0..9 with 0 being no compression and 9 the highest level of compression possible. Depending on how well your data compresses, this could well end up more than compensating for Borg’s inability to let machines write to the same repository.
In our experience Borg also has a significant advantage when it comes to performance. If we perform similar backups to the same remote destination over SSH (Borg) and SFTP (Restic), the initial backups take roughly the same time, but the subsequent incremental backups take something like 10x longer with Restic. CPU usage appears to be about the same, but Restic uses much more memory than Borg. We have seen a number of cases where Restic has been stopped by the OOM killer before backup of a small VPS could complete.
Borg and Restic both support pruning backups as they age, e.g. you could decide to keep hourly backups from the last 24 hours, daily backups from the last two weeks, the last 6 monthly backups, etc. With Borg this operation only takes a few moments to complete, but it can take several hours to prune a Restic repository (and a lot of memory). Worse, while Restic is completing the prune operation, it places an exclusive lock on the repository, making any attempts to backup to that repository fail. This is a significant limitation – especially if you backup multiple machines to the same repository.
And the Winner is …
The single big advantage that Restic has over Borg is the support for different types of backup storage. If you need anything other than local storage or remote storage accessed via SSH, then you will have to use Restic. On the other hand, if you have backup storage that is accessible over SSH, then the flexibility and performance of Borg makes it the better tool.
Restic’s memory requirements makes it unsuitable for backing up a small VPS with limited RAM, and the slow backup verification process makes it impractical on larger servers. But if you are backing up desktop or laptop computers then this may not matter so much, and using Restic means that you don’t have to setup your own storage server.