Bdtyktl

My problem is this: My laptop has a relatively slow disk subsystem (amd I'm not going to buy a better one) and I backup my system using rsync which works well for me. However, during the backup process, the files read are read into the buffer/cache of the system, which eventually triggers the swap system.

For example, running cat Win10.qcow2 > /dev/null a 60 GB file will result in

free -h

              total        used        free      shared  buff/cache   available

Mem:           15Gi       2.2Gi       210Mi       170Mi        13Gi        12Gi

Swap:          30Gi        14Mi        30Gi

and if I write to a real device, like my USB backup drive, the swap starts being used, up to a couple of GB. I do have vm.swappiness = 0 in /etc/sysctl

By itself, this is not bad, but because of my slow disk system, the computer becomes less than sprightly in response to inputs. Painfully slow, in fact.

What I would like to have is a method for limiting the amount of page buffer that the process can consume, leaving enough room to run smaller commands, such as opening a terminal.

What I have tried, is using lxc, which did not limit the system use of buffers, docker which I could not fully figure out yet, and I'm attempting to get lxd running, but I'll need some time to figure that one out.

There is a program nocache which I think works, but rsync then does not output progress indicators.

By default when rsync updates your backup it creates a copy of the file and then moves it into place. To avoid this step you can have rsync write directly to your backup with the --inplace argument.

As per https://linux.die.net/man/1/rsync:

--inplace

This option changes how rsync transfers a file when the file's data needs to be updated: instead of the default method of creating a
new copy of the file and moving it into place when it is complete,
rsync instead writes the updated data directly to the destination
file.

This has several effects:

(1) in-use binaries cannot be updated
(either the
OS will prevent this from happening, or binaries that attempt to swap-in their data will misbehave or crash),

(2) the file's data will
be in an inconsistent state during the transfer,

(3) a file's data may
be left in an inconsistent state after the transfer if the transfer is
interrupted or if an update fails,

(4) a file that does not have write
permissions can not be updated, and

(5) the efficiency of rsync's
delta-transfer algorithm may be reduced if some data in the
destination file is overwritten before it can be copied to a position
later in the file (one exception to this is if you combine this option
with --backup, since rsync is smart enough to use the backup file as
the basis file for the transfer).

WARNING: you should not use this
option to update files that are being
accessed by others, so be careful when choosing to use this for a copy.

This option is useful for transfer of large files with
block-based changes
or appended data, and also on systems that are disk bound, not network bound.

The option implies --partial (since an interrupted
transfer does not delete
the file), but conflicts with --partial-dir and --delay-updates. Prior to rsync 2.6.4 --inplace was also incompatible with
--compare-dest and --link-dest.