TrueCloudLab/restic
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

add --disable-scanner to backup command

Browse source 
The scanner process has only cosmetic effect for the progress printer,
and can be disabled without impacting functionality when the user does
not need an estimate of completion.

In many cases the scanner process can provide beneficial priming of
the file system cache, so as general advice it should not be disabled.
However, tests have shown that backup of NFS and fuse based filesystems,
where stat(2) is relatively expensive, can be significantly faster
without the scanner.
This commit is contained in:
Kjetil Torgrim Homme 2022-09-14 17:20:11 +02:00 committed by Michael Eischer
parent 7bdb985dde
commit 14aa6f2a00
3 changed files with 30 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
changelog/unreleased/pull-3931 Normal file
View file

@ -0,0 +1,8 @@
Enhancement: Allow backup file tree scanner to be disabled
Restic walks the file tree in a separate scanner process to find the total size
and file/directory count, and uses that to provide an ETA. This can slow down
backups, especially of network filesystems. The new flag `--no-scan`
can be used to speed up such backups.
https://github.com/restic/restic/pull/3931

20
cmd/restic/cmd_backup.go
View file

@ -99,6 +99,7 @@ type BackupOptions struct {
UseFsSnapshot bool
DryRun bool
ReadConcurrency uint
NoScan bool
}
var backupOptions BackupOptions
@ -138,6 +139,7 @@ func init() {
f.BoolVar(&backupOptions.IgnoreInode, "ignore-inode", false, "ignore inode number changes when checking for modified files")
f.BoolVar(&backupOptions.IgnoreCtime, "ignore-ctime", false, "ignore ctime changes when checking for modified files")
f.BoolVarP(&backupOptions.DryRun, "dry-run", "n", false, "do not upload or write any data, just show what would be done")
f.BoolVar(&backupOptions.NoScan, "no-scan", false, "do not run scanner to estimate size of backup")
if runtime.GOOS == "windows" {
f.BoolVar(&backupOptions.UseFsSnapshot, "use-fs-snapshot", false, "use filesystem snapshot where possible (currently only Windows VSS)")
}
@ -586,16 +588,18 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
targets = []string{filename}
}
sc := archiver.NewScanner(targetFS)
sc.SelectByName = selectByNameFilter
sc.Select = selectFilter
sc.Error = progressPrinter.ScannerError
sc.Result = progressReporter.ReportTotal
if !opts.NoScan {
sc := archiver.NewScanner(targetFS)
sc.SelectByName = selectByNameFilter
sc.Select = selectFilter
sc.Error = progressPrinter.ScannerError
sc.Result = progressReporter.ReportTotal
if !gopts.JSON {
progressPrinter.V("start scan on %v", targets)
if !gopts.JSON {
progressPrinter.V("start scan on %v", targets)
}
wg.Go(func() error { return sc.Scan(cancelCtx, targets) })
}
wg.Go(func() error { return sc.Scan(cancelCtx, targets) })
arch := archiver.New(repo, targetFS, archiver.Options{ReadConcurrency: backupOptions.ReadConcurrency})
arch.SelectByName = selectByNameFilter

11
doc/040_backup.rst
View file

@ -204,8 +204,17 @@ Combined with ``--verbose``, you can see a list of changes:
modified /archive.tar.gz, saved in 0.140s (25.542 MiB added)
Would be added to the repository: 25.551 MiB
.. _backup-excluding-files:
Disabling Backup Progress Estimation
************************************
When you start a backup, restic will concurrently count the number of
files and their total size, which is used to estimate how long it will
take. This will cause some extra I/O, which can slow down backup of
network file systems or fuse mounts.
- ``--no-scan`` Do not run scanner to estimate size of backup
.. _backup-excluding-files:
Excluding Files
***************