At another page, I have described how I backup Windows machines. Under Windows, I prefer image backups because the integration between user data and the OS is tight, especially for user settings. Back-up under Windows is simplified by the fact that volume-shadow-copy technology is built-in, which means that you can back-up a running system.
When installing Debian Linux, I had decided to keep things simple and not use LVM (Logical Volume Manager). LVM offers similar functionality to Windows shadow copy. E.g. LVM allows to back-up a system while it is running.
When not using LVM, it may seem you have to boot the PC from an external medium in order to back-up, because only then you can be sure there are no files in an intermediate state (especially data bases).
However, when using the flexibility Linux offers, you can avoid that for the most part:
sudo, because root's home directory is /root, hence this directory is not part of the /home directory tree/home directory/home as by the "User Private Groups" configuration that Debian Linux supports/home on a separate disk partition
/home read-only before back-up to make sure that no write access takes place while back-up is running/home is quite large, I put it even on a separate disk driveHence I do two separate and different back-up processes:
/home as root, while the PC is booted normally, but only root is logged in
I use the tar program for file backups, because
To make backing-up as easy as possible and to avoid manual errors, I developed a shell script, that automates the back-up process. You can download the script as part of the file ./tr-lnx-bckp.zip
The file ./tr-lnx-bckp.zip contains the following files:
| File Name | Comment |
|---|---|
readme.txt |
Text version of this web site |
tr-tar-backup |
Shell script for file based backup using tar |
tr-pcl-backup |
Shell script for image backup using partclone |
tr-tar-backup in a convenient location, e.g. in /root/bin, and make it executable using chmodtr-tar-backup for detailsroot starts the script tr-tar-backuptar will write a message to the console showing the amount of data written and the average speedtr-tar-backup for the success or error messageroot shuts the PC down or re-mounts the drive containing /home such that normal operations could resumetr-tar-backup Functionality
The script tr-tar-backup provides the following functionality:
MYBCKPnn
nn stands for Arabic numerals which you can use to discern multiple back-up target diskse2label for ext2/3/4 partitions, ntfslabel for NTFS partitions, fatlabel for FAT partitionstr-tar-backup supports backing up multiple PC. This is achieved by manually creating appropriate sub-directories on the back-up target disk(s) that match the PC's hostname.
The script tr-tar-backup implements a simple two set rotation scheme. A backup target disk shall have two target directories: A and B. The purpose is to always have two full back-ups available. The two sets rotate each month. In the same month, further back-ups are run as "differential" back-ups. The frequency of the differential back-ups is up to the (root) user, e.g. weekly, or daily, or when required.
The following graphics visualise this rotation scheme. Time runs from left to right. The blue bars symbolise the back-up data. The longer thicker bars stand for full back-ups, the thinner shorter bars stand for differential back-ups. Normally, differential back-ups will get bigger over the course of a month:
Note that after two months, the first set gets deleted, such that only two months of back-ups are available at any point in time.
The scheme implemented by tr-tar-backup allows for a longer back-up history, when you add another back-up drive. This is visualised in the following graphics:
A second drive with partition label MYBCKP02 gets only connected to the PC every six months. It will thus contain two full back-ups which reach up to one year back in time. Note that this drive will never contain differential back-ups because of the month logic in the tr-tar-backup script.
Of course you could connect a third drive to receive full back-ups. Such a drive could be stored safely at a different building.
tr-tar-backup described in the previous section only backs up /home. The file ./tr-lnx-bckp contains another script tr-pcl-backup. It employs the partclone tool (Debian package partclone) to do an image back-up of the system-disk.
tr-pcl-backup needs to be started from a disk that is different from the system disk to be backed-up. The simplest method to achieve this is to temporarily connect a drive to the PC via USB and boot the PC from that drive.
For this purpose, I generated a small live Debian Linux system using the online tool https://fai-project.org/FAIme/live/. I stored the script file tr-pcl-backup on a separate "DATA" partition on the USB stick. The FAI tool offers to create such a partition. However, any live Linux system could be used to run tr-pcl-backup. The only requirement is, that partclone is already part of the live system or can be added without much effort.
The image back-up does not take long, because the system disk of my PC only uses about 25 GB. Hence I did not feel the need to implement a rotation scheme, i.e. tr-pcl-backup always creates a full image back-up.
tr-pcl-backup can also work with multiple back-up target drives.
Caveat: tr-pcl-backup currently supports only one PC to be backed up, because the PC's hostname is hardwired in the script. I will amend that in a future version of the script.
tar
tar, whose name comes from "tape archive", is an old and proven utility. It is available for all Unix-like operating systems.
There seem to be two major tar variants:
tar. This is available on most Linux systems. However, there is also a port available for Windows. For example, I got GNU tar version 1.30 from 2017 as part of git for Windows.tar. This is available on systems like FreeBSD, but has been part of Mac OS. Since Windows 10 1903, it is also part of the standard Windows command line tools
tar is mostly used to do backups. It used to be used to do backups on physical tapes. The functionality still reflects limitations of tapes. For example the tar "update" mode will write the updated file(s) to the end of the archive, while leaving the older information in the archive untouched. This is because "seeking" on a tape was slow and costly.
To do incremental backups, you might consider using the "update" mode, to add new or updated files to the archive. This mode is supported by both BSD tar and GNU tar.
To restore, you would use the "extract" mode together with option -k "Keep (don't overwrite) existing files". However, this approach does not cater for files that were deleted in the source between the full backup and subsequent updates. Also files that were renamed would appear twice after extract:
Similar for files that were moved: they appear
Another drawback of "update" is the fact, that it works only on uncompressed archives. This is due to the fact that tar compresses after creating the archive and before writing to tape/file. The approach seems to be due to legacy constraints:
GNU tar has additional functionality for incremental backups. This is not available in BSD tar. GNU tar creates and maintains an additional file with meta data along side the archive file. The standard extension for this file is snar. Example syntax:
tar -czvg mybackup_full.snar -f mybackup_full.tgz dir-to-backup
The options
c stands for "create"z for gzip compressionv for verbose messagesg is short hand for --listed-incremental.
An additional option p ‑ to preserve meta data like access rights ‑ is only relevant for extract.
There is no mention of the "differential backup" strategy in the GNU tar documentation. However, the GNU tar manual (see section 5.) calls a full backup a "level 0" backup and the first incremental backup "level 1". The manual explicitly states that you might want to do more level 1 backups by creating a "working copy" of the snar file created at level 0. I tend to call this approach a differential backup. The approach would be:
cp mybackup_full.snar mybackup_diff-1.snar tar -czvg mybackup_diff-1.snar -f mybackup_diff-1.tgz dir-to-backup
To restore or "extract" from incremental backups, the GNU tar manual stipulates to also use option --listed-incremental with =/dev/null as argument (it could be any argument). So with just a full and a level 1 backup:
tar -xzpf mybackup_full.tgz tar -xzpg /dev/null -f mybackup_diff-1.tgz
The following alternative also works:
tar --incremental -xzpf mybackup_diff-1.tgz
To list what is going on, you might do:
tar --incremental -tvvzpf mybackup_diff-1.tgz
tar stores "device numbers" in the snapshot file and also uses these to check if a file has changed since the last backup. You might want to use --no-check-device option to avoid a full backup if device numbers change for some reason. There is a tar-snapshot-edit (tar-edit-snapshot ?) utility to deal with such snapshot file issues.
tar manual
https://www.gnu.org/savannah-checkouts/gnu/tar/manual/tar.html#Backupssnar file
https://etutorials.org/Linux+systems/how+linux+works/Chapter+13+Backups/13.4+Using+tar+for+Backups+and+Restores/--listed-incremental and --newer
https://unix.stackexchange.com/questions/307530/what-is-the-difference-between-tars-newer-and-listed-incremental-options
Home
Projects
Documentation
WSL