This article will cover software tools for backup that create a repository by splitting data streams into separate components (chunks).
The repository components can be further compressed and encrypted, and most importantly — reused during subsequent backup processes.
A backup in such a repository is a named chain of interrelated components, for example, based on various hash functions.
There are several such solutions; I will focus on three: zbackup, borgbackup, and restic.
Expected Results
Since all candidates require creating a repository in one way or another, one of the most important factors will be assessing the repository size. Ideally, its size should not exceed 13 GB according to the accepted methodology, or even less—with good optimization.
It is also highly desirable to have the ability to create backups of files directly, without using archivers like tar, as well as to work with ssh/sftp without additional tools like rsync and sshfs.
Behavior during Backup Creation:
- The size of the repository will equal the size of the changes, or be smaller.
- A high CPU load is expected when using compression and/or encryption, as well as potentially significant load on the network and disk subsystem if the archiving and/or encryption process runs on the backup storage server.
- If the repository is corrupted, a delayed error is likely both during the creation of new backups and during restoration attempts. Additional measures to ensure the integrity of the repository need to be planned, or built-in integrity checking tools should be used.
The reference value is working with tar, as shown in a previous article.
Testing zbackup
The overall mechanism of how zbackup works is that the program finds areas in the input data stream that contain identical data, then optionally compresses and encrypts them, saving each area only once.
A 64-bit circular hash function with a sliding window is used for deduplication to check byte-by-byte for matches with existing data blocks (similar to how it is implemented in rsync).
LZMA and LZO are applied for compression in a multithreaded manner, while AES is used for encryption. In the latest versions, there is an option to delete old data from the repository in the future.
The program is written in C++ with minimal dependencies. The author seems to have been inspired by the Unix philosophy, as the program accepts input on stdin when creating backups and outputs a similar data stream on stdout when restoring. Thus, zbackup can serve as a solid building block for custom backup solutions. For instance, the author of this article has been using this program as the primary backup tool for personal machines since about 2014.
The data stream will use the standard tar, unless specified otherwise.
Let's see what the results will be:
The testing was conducted in 2 variations:
- a repository is created and zbackup is run on the server with the source data, then the contents of the repository are transferred to the backup storage server.
- a repository is created on the backup storage server, zbackup is launched via SSH on the backup storage server, receiving data from the pipe.
The results of the first option were as follows: 43m11s — with an unencrypted repository and LZMA compressor, 19m13s — when switching the compressor to LZO.
The load on the server with the source data was as follows (an example with LZMA is shown; with LZO, the picture was approximately the same, but the rsync share was about a quarter of the time):
It is clearly evident that such a backup process is only suitable for relatively rare and minor changes. It is also highly advisable to limit the operation of zbackup to 1 thread; otherwise, CPU load will be significantly high, as the program is quite adept at working with multiple threads. The disk load was low, which, given the modern SSD-based disk subsystem, will generally go unnoticed. It is also clearly visible that the data synchronization process to the remote server is underway, with speeds comparable to regular rsync, limited by the performance of the backup server's disk subsystem. A downside of this approach is the storage of a local repository, which consequently leads to data duplication.
A more interesting and practical alternative is the second option of running zbackup directly on the backup storage server.
To start, the operation will be tested without using encryption with the lzma compressor:
Execution time of each test run:
Run 1
Run 2
Run 3
39m45s
40m20s
40m3s
7m36s
8m3s
7m48s
15m35s
15m48s
15m38s
If encryption using aes is activated, the results are quite close:
Running time on the same data, with encryption:
Run 1
Run 2
Run 3
43m40s
44m12s
44m3s
8m3s
8m15s
8m12s
15m0s
15m40s
15m25s
If encryption is combined with lzo compression, it appears as follows:
Running time:
Run 1
Run 2
Run 3
18m2s
18m15s
18m12s
5m13s
5m24s
5m20s
8m48s
9m3s
The size of the resulting repository was relatively consistent at 13GB. This means that deduplication works correctly. Additionally, applying lzo to already compressed data yields a noticeable effect, causing the total running time of zbackup to closely approach duplicity/duplicati, although it lags behind those based on librsync by 2-5 times.
The advantages are obvious — saving disk space on the backup storage server. As for repository verification tools, they are not provided by the zbackup author, and it is recommended to use a fault-tolerant disk array or cloud provider.
Overall, a rather pleasant impression, despite the fact that the project has been stagnant for about 3 years (the last feature request was made around a year ago, but without a response).
Testing borgbackup
Borgbackup is a fork of attic, another similar system to zbackup. Written in Python, it has a similar list of features to zbackup, but additionally can:
BorgBackup is a fork of Attic, another system similar to zbackup. Written in Python, it has a feature set comparable to zbackup, but additionally it can:
- Mount backups via fuse
- Check the contents of the repository
- Operate in client-server mode
- Use various data compressors as well as heuristic file type detection during compression.
- 2 encryption options, aes and blake
- Built-in tool for
performance testing
borgbackup benchmark crud ssh://backup_server/repo/path local_dir
The results were as follows:
C-Z-BIG 96.51 MB/s (10 100.00 MB all-zero files: 10.36s)
R-Z-BIG 57.22 MB/s (10 100.00 MB all-zero files: 17.48s)
U-Z-BIG 253.63 MB/s (10 100.00 MB all-zero files: 3.94s)
D-Z-BIG 351.06 MB/s (10 100.00 MB all-zero files: 2.85s)
C-R-BIG 34.30 MB/s (10 100.00 MB random files: 29.15s)
R-R-BIG 60.69 MB/s (10 100.00 MB random files: 16.48s)
U-R-BIG 311.06 MB/s (10 100.00 MB random files: 3.21s)
D-R-BIG 72.63 MB/s (10 100.00 MB random files: 13.77s)
C-Z-MEDIUM 108.59 MB/s (1000 1.00 MB all-zero files: 9.21s)
R-Z-MEDIUM 76.16 MB/s (1000 1.00 MB all-zero files: 13.13s)
U-Z-MEDIUM 331.27 MB/s (1000 1.00 MB all-zero files: 3.02s)
D-Z-MEDIUM 387.36 MB/s (1000 1.00 MB all-zero files: 2.58s)
C-R-MEDIUM 37.80 MB/s (1000 1.00 MB random files: 26.45s)
R-R-MEDIUM 68.90 MB/s (1000 1.00 MB random files: 14.51s)
U-R-MEDIUM 347.24 MB/s (1000 1.00 MB random files: 2.88s)
D-R-MEDIUM 48.80 MB/s (1000 1.00 MB random files: 20.49s)
C-Z-SMALL 11.72 MB/s (10000 10.00 kB all-zero files: 8.53s)
R-Z-SMALL 32.57 MB/s (10000 10.00 kB all-zero files: 3.07s)
U-Z-SMALL 19.37 MB/s (10000 10.00 kB all-zero files: 5.16s)
D-Z-SMALL 33.71 MB/s (10000 10.00 kB all-zero files: 2.97s)
C-R-SMALL 6.85 MB/s (10000 10.00 kB random files: 14.60s)
R-R-SMALL 31.27 MB/s (10000 10.00 kB random files: 3.20s)
U-R-SMALL 12.28 MB/s (10000 10.00 kB random files: 8.14s)
D-R-SMALL 18.78 MB/s (10000 10.00 kB random files: 5.32s)
During testing, heuristic compression with file type detection (compression auto) will be used, and the results will be:
Let's first check the operation without encryption:
Running time:
Run 1
Run 2
Run 3
4m6s
4m10s
4m5s
56s
58s
54s
1m26s
1m34s
1m30s
If repository authorization is enabled (authenticated mode), the results will be similar:
Running time:
Run 1
Run 2
Run 3
4m11s
4m20s
4m12s
1m0s
1m3s
1m2s
1m30s
1m34s
1m31s
When activating aes encryption, the results did not worsen significantly:
Run 1
Run 2
Run 3
4m55s
5m2s
4m58s
1m0s
1m2s
1m0s
1m49s
1m50s
1m50s
And if we change aes to blake, the situation will improve:
Running time:
Run 1
Run 2
Run 3
4m33s
4m43s
4m40s
59s
1m0s
1m0s
1m38s
1m43s
1m40s
As with zbackup, the repository size was 13 GB and even a bit less, which is generally expected. The runtime was quite impressive, comparable to solutions based on librsync, providing much broader capabilities. I was also pleased by the ability to set various parameters via environment variables, which gives a significant advantage when using borgbackup in automatic mode. Furthermore, the load during backup was notable: judging by the CPU load — borgbackup operates in a single thread.
No significant downsides were found during use.
Testing restic
Despite being a relatively new solution (the first two candidates were known since 2013 or earlier), restic boasts quite decent characteristics. It is written in Go.
Compared to zbackup, it additionally offers:
- Integrity checks for the repository (including partial checks).
- A huge list of supported protocols and providers for backup storage, as well as support for rclone—rsync for cloud solutions.
- Comparison of two backups against each other.
- Mounting the repository via fuse.
Overall, the list of features is quite similar to borgbackup, sometimes more, sometimes less. One distinct feature is the inability to disable encryption, meaning backups will always be encrypted. Let's see in practice what can be extracted from this software:
The results turned out as follows:
Running time:
Run 1
Run 2
Run 3
5m25s
5m50s
5m38s
35s
38s
36s
1m54s
2m2s
1m58s
The performance results are also comparable with solutions based on rsync and are generally quite close to borgbackup, but the CPU load is higher (multiple threads are running) and spiky.
Most likely, the program is limited by the performance of the disk subsystem on the data storage server, as was already the case with rsync. The size of the repository was 13 GB, similar to zbackup or borgbackup, and no obvious downsides were found when using this solution.
Results
In fact, all candidates produced similar results, albeit at different prices. Borgbackup performed the best, with restic slightly slower, whereas zbackup might not be worth starting to use,
and if it's already in use—consider switching to borgbackup or restic.
Conclusions
Restic appears to be the most promising solution, as it has the best balance of features to speed, but let's not rush to any conclusions just yet.
Borgbackup is essentially just as good, but zbackup is likely better replaced. However, to adhere to the 3-2-1 rule, zbackup can still be utilized. For instance, as an addition to backups based on (lib)rsync.
Announcement
Backup, Part 5: Testing Bacula and Veeam Backup for Linux
Backup, Part 6: Comparison of Backup Solutions
Backup, Part 7: Conclusions
Author: Pavel Demkovich
Source: habr.com
