Alexander Weiss
bce87922c0
Improve recover
...
- only save directories not referenced by any snapshot
- dont't write empty snapshot
- use progress bar
2021-09-03 21:36:57 +02:00
Alexander Weiss
81876d5c1b
Simplify cache logic
2021-09-03 21:01:00 +02:00
greatroar
7f0aa49f45
cmd/restic: Streamline progress printing
...
* PrintProgress no longer does unnecessary Sprintf calls, and performs
fewer allocations in general
* newProgressMax's callback checks whether the terminal supports
line updates once instead of once per call
* the callback looks up the terminal width once per call instead of
twice (on Windows)
* the status shortening now uses the Unicode-aware version from
internal/ui/termstatus (future-proofing)
2021-09-03 11:48:22 +02:00
Juergen Hoetzel
409e4936af
Improve snapshot filter warning message
...
Include the root-cause why the snapshot prefix is ignored.
2021-08-25 15:46:21 +02:00
Michael Eischer
877fc9f352
rebuild-index: test that invalid indexes are skipped and removed
2021-08-22 18:24:19 +02:00
Michael Eischer
64258a2c2a
rebuild-index: Ignore broken index files
...
Previously, an e.g. truncated index could prevent rebuild-index from
working.
2021-08-22 18:23:47 +02:00
Alexander Neumann
d90efd7704
Fix test
2021-08-09 10:30:10 +02:00
Alexander Neumann
9fe5a87785
Merge pull request #3429 from MichaelEischer/safe-key-change
...
key: Check that new key works before deleting the old one
2021-08-09 10:07:15 +02:00
erin
4126435663
resolve rawtaz's review comments
...
make majority of suggestions from review by @rawtaz verbatim, with one clarification on my part in changelog
2021-08-04 21:19:29 +02:00
Alexander Weiss
d107a2cfdf
Separate dry run tests
2021-08-04 21:19:29 +02:00
Ryan Hitchman
77bf148460
backup: add --dry-run/-n flag to show what would happen.
...
This can be used to check how large a backup is or validate exclusions.
It does not actually write any data to the underlying backend. This is
implemented as a simple overlay backend that accepts writes without
forwarding them, passes through reads, and generally does the minimal
necessary to pretend that progress is actually happening.
Fixes #1542
Example usage:
$ restic -vv --dry-run . | grep add
new /changelog/unreleased/issue-1542, saved in 0.000s (350 B added)
modified /cmd/restic/cmd_backup.go, saved in 0.000s (16.543 KiB added)
modified /cmd/restic/global.go, saved in 0.000s (0 B added)
new /internal/backend/dry/dry_backend_test.go, saved in 0.000s (3.866 KiB added)
new /internal/backend/dry/dry_backend.go, saved in 0.000s (3.744 KiB added)
modified /internal/backend/test/tests.go, saved in 0.000s (0 B added)
modified /internal/repository/repository.go, saved in 0.000s (20.707 KiB added)
modified /internal/ui/backup.go, saved in 0.000s (9.110 KiB added)
modified /internal/ui/jsonstatus/status.go, saved in 0.001s (11.055 KiB added)
modified /restic, saved in 0.131s (25.542 MiB added)
Would add to the repo: 25.892 MiB
2021-08-04 21:19:29 +02:00
Alexander Neumann
7049f1cbfc
Set development version for 0.12.1
2021-08-03 11:45:39 +02:00
Alexander Neumann
dc7a8aab24
Add version for 0.12.1
2021-08-03 11:45:36 +02:00
Alexander Neumann
77551597b2
Merge pull request #3416 from torfason/keep-hourly-within
...
Keep hourly within
2021-07-30 10:36:41 +02:00
Alexander Neumann
92f293cd0b
Merge pull request #3427 from MichaelEischer/find-packs-from-index
...
find: List missing pack files based on the index
2021-07-30 10:33:02 +02:00
Magnus Thor Torfason
74ebc650ab
forget: Add --keep-within-hourly (and friends)
...
Allow keeping hourly/daily/weekly/monthly/yearly snapshots for a given time period.
This adds the following flags/parameters to restic forget:
--keep-within-hourly duration
--keep-within-daily duration
--keep-within-weekly duration
--keep-within-monthly duration
--keep-within-yearly duration
Includes following changes:
- Add tests for --keep-within-hourly (and friends)
- Add documentation for --keep-within-hourly (and friends)
- Add changelog for --keep-within-hourly (and friends)
2021-07-24 16:14:43 +00:00
Alexander Neumann
c707d71b72
Merge pull request #3401 from MichaelEischer/goroutine-shutdown-cleanups
...
Goroutine shutdown cleanups
2021-07-11 16:32:28 +02:00
Michael Eischer
0880afe67b
Use our generate command instead of cobra's complete command
2021-07-10 19:44:18 +02:00
Michael Eischer
3442dc87fb
find: Address review comments
2021-07-06 21:04:34 +02:00
Michael Eischer
95b44490a0
find: search blob ids for pack in index if pack is missing
...
If a pack file is missing try to determine the contained pack ids based
on the repository index. This helps with assessing the damage to a
repository before running `rebuild-index`.
2021-07-06 21:04:34 +02:00
Michael Eischer
3caab3c7ac
find: Print not found pack files
2021-07-06 21:04:34 +02:00
Michael Eischer
40745b4f82
find: stop file listing after resolving all pack files
2021-07-06 21:04:34 +02:00
Michael Eischer
6c01078f3d
find: support resolving multiple pack ids to blobs
...
Just passing the list of blobs to packsToBlobs would also work in most
cases, however, it could cause unexpected results when multiple pack
files have the same prefix. Forget found prefixes to prevent this.
2021-07-06 21:04:34 +02:00
Michael Eischer
aad8864835
prune: Add missing newlines in error descriptions
2021-06-20 14:25:40 +02:00
Michael Eischer
454b6d608e
key: Add test that failed key saves don't break the repository
2021-06-13 13:46:48 +02:00
Michael Eischer
6add186867
key: Check that a new key file actually works
2021-06-12 23:09:19 +02:00
Michael Eischer
fe43f53528
mount: enable fuse readahead
...
Apparently readahead was disabled by default. Enable readahead with the
Linux default size of 128kB. Larger values seem to have no effect.
This can speed up reading from the fuse mount by at least factor 5.
Speedup for a 1G random file stored in a local repository:
(Only one result shown, but times were quite stable, restarted restic
after each command)
$ dd if=/dev/urandom bs=1M count=1024 of=rand
$ shasum -a 256 tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d tmp/rand
before: $ time shasum -a 256 fuse/snapshots/latest/tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d fuse/snapshots/latest/tmp/rand
real 0m18.294s
user 0m4.522s
sys 0m3.305s
before: $ time cat fuse/snapshots/latest/tmp/rand > /dev/null
real 0m14.924s
user 0m0.000s
sys 0m4.625s
after: $ time shasum -a 256 fuse/snapshots/latest/tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d fuse/snapshots/latest/tmp/rand
real 0m6.106s
user 0m3.115s
sys 0m0.182s
after: $ time cat fuse/snapshots/latest/tmp/rand > /dev/null
real 0m3.096s
user 0m0.017s
sys 0m0.241s
2021-06-12 17:07:30 +02:00
rawtaz
fdbd65485e
Merge pull request #3402 from MichaelEischer/misc-fixes
...
Various small code cleanups
2021-05-24 11:30:31 +02:00
rawtaz
2daf033156
Merge pull request #3403 from MichaelEischer/fast-cat
...
cat: only load index if really necessary
2021-05-24 11:02:10 +02:00
MichaelEischer
5dad45f005
Merge pull request #3312 from soraxas/master
...
Bump cobra and add completions for fish
2021-05-23 13:58:27 +02:00
Michael Eischer
61b368ddea
cat: only load index if really necessary
2021-05-23 13:11:55 +02:00
Michael Eischer
fd8bce8184
backup: cleanly shutdown goroutines on error
2021-05-23 13:02:44 +02:00
greatroar
d7322a5f36
restic ls --json: print "size":0 for empty files
2021-05-21 21:06:00 +02:00
Tin Lai
9cc1ecdd45
bump cobra and add completions for fish
...
Signed-off-by: Tin Lai <oscar@tinyiu.com>
2021-05-21 13:47:52 +10:00
Alexander Neumann
af3de702c7
Merge pull request #3332 from restic/debug-1999
...
Merge `debug examine` to salvage damaged pack files
2021-05-18 09:38:40 +02:00
Alexander Neumann
226cd8d4d1
Merge pull request #3310 from MichaelEischer/copy-unstable-tree
...
`copy` raw tree blobs
2021-05-18 09:36:51 +02:00
Alexander Neumann
4cabad8c34
Merge pull request #3325 from MichaelEischer/fix-mintty-output
...
Fix windows terminal output for mintty
2021-05-18 09:29:24 +02:00
Michael Eischer
5767c65c62
delete: properly close fileChan if context is canceled
2021-05-17 21:05:54 +02:00
Michael Eischer
5a87a0ba0a
find: use Str() to format short ids
2021-05-17 20:56:49 +02:00
Leo R. Lundgren
90d75651e6
backup: Improve wording for --one-file-system description
2021-05-15 00:06:27 +02:00
Sébastien Gross
2a92b68e65
cmd/snapshots: Add option to limit snapshots list
...
This patch adds a `--latest` option to limit snapshots list to the n
last snapshots. It is very similar to the `--last` one but does not
limit to one entry. It also deprecates the `--last` flag usage in
favor of `--latest 1`
Output example:
$ restic snapshots --latest 2
repository 0d3eb989 opened successfully, password is correct
ID Time Host Tags Paths
------------------------------------------------------------
5a33bdcc 2020-12-14 12:30:00 local /home
73887d8e 2020-12-15 12:30:00 local /home
------------------------------------------------------------
2 snapshots
Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2021-05-13 20:18:23 +02:00
Michael Eischer
5e6af77b7a
Unify interactive terminal detection code
...
Previously the progress bar / status update interval used
stdoutIsTerminal to determine whether it is possible to update the
progress bar or not. However, its implementation differed from the
detection within the backup command which included additional checks to
detect the presence of mintty on Windows. mintty behaves like a terminal
but uses pipes for communication.
This adds stdoutCanUpdateStatus() which calls the same terminal detection
code used by backup. This ensures that all commands consistently switch
between interactive and non-interactive terminal mode.
stdoutIsTerminal() now also returns true whenever stdoutCanUpdateStatus()
does so. This is required to properly handle the special case of mintty.
2021-04-11 20:02:09 +02:00
MichaelEischer
efb10b3c40
Merge pull request #3347 from jazcap53/fix_ambiguous_warning
...
Change ambiguous Warning message
2021-03-27 12:54:07 +01:00
Andrew Jarcho
9f44129c2f
Change ambiguous Warning message
...
Fixes #3346
On branch fix_ambiguous_warning
Changes to be committed:
modified: cmd_backup.go
2021-03-24 10:44:47 -04:00
Hendrik Luup
5592c17e4a
Display 'created new cache in ' message only if output is terminal
2021-03-22 11:31:08 +02:00
Michael Eischer
54d58edacc
debug: fix usage for examine command
2021-03-10 22:22:33 +01:00
Michael Eischer
5975ed61f3
debug: fix linter warning
2021-03-10 21:22:53 +01:00
Michael Eischer
dc62ec5933
debug: use Printf/Warnf for output
2021-03-10 21:20:21 +01:00
Michael Eischer
547d9b384d
debug: cleanup repair code
2021-03-10 21:15:38 +01:00
Michael Eischer
fa7b9d5dfe
debug: Cleanup pack size checks
2021-03-10 20:57:14 +01:00
Michael Eischer
6774fc6454
debug: check arguments and cleanup error handling
2021-03-10 20:43:22 +01:00
Michael Eischer
096f15db5c
debug: extract examinePack function
2021-03-10 20:21:05 +01:00
Michael Eischer
84491ff40b
debug: store repaired and correct blobs
2021-03-10 20:03:44 +01:00
Alexander Neumann
b3c3121622
debug: Save raw decrypt (disregarding signature)
2021-03-10 20:03:44 +01:00
Alexander Neumann
ce4b6d0874
examine: add byte repair mode
2021-03-10 20:03:44 +01:00
Alexander Neumann
52061e817c
debug: fix percentage
2021-03-10 20:03:44 +01:00
Alexander Neumann
133ac42a0b
debug: check file content hash
2021-03-10 20:03:44 +01:00
Alexander Neumann
90f975fa1c
debug: make output less verbose
2021-03-10 20:03:44 +01:00
Alexander Neumann
086993bae1
debug: check packs not in index, implement repair
2021-03-10 20:03:44 +01:00
Alexander Neumann
d6f78163d4
Add 'debug examine' command to debug #1999
2021-03-10 20:03:44 +01:00
MichaelEischer
c9b4fadd91
Merge pull request #3294 from Achilleshiel/fix-copy-repofile
...
Add RepositoryFile2 Option for secondary repository
2021-03-08 22:50:31 +01:00
Michael Eischer
da458a55db
Cleanup comments in secondary repo test
2021-03-08 22:41:13 +01:00
Wouter Horlings
9ccdba9df6
Add repositoryFile2 option
...
The `init` and `copy` commands can now use `--repository-file2` flag and
the `$RESTIC_REPOSITORY_FILE2` environment variable.
This also fixes the conflict with the `--repository-file` and `--repo2`
flag.
Tests are added for the initSecondaryGlobalOpts function.
This adds a NOK function to the test helper functions. This NOK tests if
err is not nil, and otherwise fail the test.
With the NOK function a couple of sad paths are tested in the
initSecondaryGlobalOpts function.
In total the tests checks wether the following are passed correct:
- Password
- PasswordFile
- Repo
- RepositoryFile
The following situation must return an error to pass the test:
- no Repo or RepositoryFile defined
- Repo and RepositoryFile defined both
2021-03-08 22:41:13 +01:00
Michael Eischer
427781928f
copy: test that trees with unstable json encoding are properly copied
2021-03-08 22:16:48 +01:00
Michael Eischer
2fc7abac35
copy: copy raw bytes of tree blobs
...
This avoids problems when for some reason the JSON encoding changes.
This also ensures forward compatibility with future restic versions
which might e.g. add new fields to the tree metadata.
2021-03-08 22:16:48 +01:00
Juergen Hoetzel
18fccb5995
backup: In case of error also Wait() for terminal goroutine
...
This prevents a race condition where the final summary message can get lost.
2021-03-08 22:11:34 +01:00
MichaelEischer
814a399e4c
Merge pull request #3283 from dennypage/master
...
Treat an empty password as a fatal error for repository init.
2021-02-28 00:49:21 +01:00
Denny Page
0f41e99ea7
Treat an empty password as a fatal error for repository init.
2021-02-28 00:40:00 +01:00
Michael Eischer
9e852af5be
check: Fix crash of --read-data-subset=x% on empty repository
...
Rounding up to at least one pack file should only be done if at least a
pack file exists.
2021-02-27 16:05:36 +01:00
Michael Eischer
65bd2a9a49
backup: Correctly handle --quiet flag
...
The quiet flag changes the backup output to assume a non-interactive
terminal. However, the output progress interval was not set to 0 by
default.
2021-02-15 22:14:58 +01:00
Alexander Neumann
bb53fcfc0d
Set development version for 0.12.0
2021-02-14 11:44:24 +01:00
Alexander Neumann
27f241334e
Add version for 0.12.0
2021-02-14 11:44:20 +01:00
Alexander Neumann
c0f2c1d871
ls: Check for non-nil error before calling panic()
2021-02-07 21:12:54 +01:00
Alexander Weiss
39a26066f7
rebuild-index: add missing bar.Done()
2021-01-31 18:28:02 +01:00
MichaelEischer
b3dc127af5
Merge pull request #3207 from aawsome/filerestorer-fix-errorhandling
...
Fix error handling in filerestorer and fix #3166
2021-01-31 17:47:46 +01:00
Alexander Weiss
6e942693ba
Fix #3166
2021-01-31 14:22:57 +01:00
Alexander Neumann
027a51529d
prune: Improve error message for missing files
...
This commit changes the error message so that a list of file names is
printed. Before, just the raw map was printed, which is not a great user
interface.
2021-01-31 11:31:27 +01:00
Alexander Neumann
f867e65bcd
Fix issues reported by staticcheck
2021-01-30 20:43:53 +01:00
Alexander Neumann
aef3658a5f
Address review comments
2021-01-30 20:02:37 +01:00
Alexander Neumann
200f09522d
Add more error checks
2021-01-30 20:02:37 +01:00
Alexander Neumann
cbd88c457a
backup: Improve error handling
2021-01-30 20:02:37 +01:00
Alexander Neumann
1a0eb05bfa
errcheck: Add more error checks
2021-01-30 20:02:37 +01:00
MichaelEischer
4d576c2f79
Merge pull request #3217 from M4a1x/read-data-subset-fix
...
Fix missing rand seed for restic check --read-data-subset=x%
2021-01-29 23:34:13 +01:00
Max Stabel
f9e1fa26ff
Fix missing rand seed for restic check --read-data-subset=x%
2021-01-29 23:18:35 +01:00
Alexander Weiss
e08e65dc30
prune: Simplify logic selecting packs to repack
2021-01-29 22:27:22 +01:00
Alexander Weiss
daeb4cdf8f
prune: Fix statistics for --repack-cacheable-only
2021-01-29 22:27:22 +01:00
Alexander Neumann
a16ce65295
Merge pull request #3244 from MichaelEischer/better-damage-reports
...
Print more details about possible repository damages
2021-01-29 11:45:45 +01:00
rawtaz
de7e3a0648
Merge pull request #2823 from greatroar/trust-mtime
...
Add --ignore-ctime flag to backup and document change detection
2021-01-29 00:02:19 +01:00
greatroar
6bd8a2faaa
backup: Add --ignore-ctime option and document change detection
2021-01-28 23:42:10 +01:00
Michael Eischer
58b5679f14
prune: reword missing blobs error
...
The previous wording could be understood such that the prune run did
damage the repository.
2021-01-28 21:48:24 +01:00
Michael Eischer
7b8886c052
prune: report missing but unneeded pack files
...
This indicates a damaged repository so add some output to help with
debugging.
2021-01-28 21:46:01 +01:00
Michael Eischer
ff95999246
rebuild-index: report added/removed/reindexed files
...
This should help with investigating missing pack files.
2021-01-28 21:46:01 +01:00
Michael Eischer
b71c52797a
find: correctly expand multiple blob ids
...
For example `restic find --show-pack-id --blob f78dc991 5b9e4366 ddd8c7d4`
would previously only expand one blob if all of them belong to the same
file.
2021-01-28 21:21:54 +01:00
MichaelEischer
82140967d3
Merge pull request #3228 from aawsome/prune-all-trees
...
prune: Remove all unused trees
2021-01-28 21:04:27 +01:00
MichaelEischer
35033d9b79
Merge pull request #3177 from MichaelEischer/fix-2759
...
prune: don't print stacktrace on console
2021-01-28 20:28:45 +01:00
Alexander Neumann
4d40c70214
Merge pull request #3211 from MichaelEischer/sftp-speedup
...
Speed-up caching/pack download via SFTP
2021-01-28 14:16:55 +01:00
Alexander Neumann
44169d0dc4
Merge pull request #3205 from MichaelEischer/fix-quiet-verbose
...
Properly check that --quiet and --verbose are not combined
2021-01-28 13:53:31 +01:00
Alexander Neumann
bdfedf1f5b
Merge pull request #3173 from MichaelEischer/unify-index-loading
...
Unify index loading
2021-01-28 13:50:42 +01:00
Michael Eischer
e2b0072441
check: add progress bar to the tree structure check
2021-01-28 11:10:50 +01:00
Michael Eischer
505f8a2229
progress/counter: Support updating the progress bar maximum
2021-01-28 11:10:47 +01:00
Michael Eischer
eda8c67616
restic: let FindUsedBlobs handle multiple snapshots at once
2021-01-28 11:08:43 +01:00
Michael Eischer
258ce0c1e5
parallel: report progress for StreamTrees
...
This assigns an id to each tree root and then keeps track of how many
tree loads (i.e. trees referenced for the first time) are pending per
tree root. Once a tree root and its subtrees were fully processed there
are no more pending tree loads and the tree root is reported as
processed.
2021-01-28 11:08:43 +01:00
Michael Eischer
3d6a3e2555
copy: Remove treeCloner struct
2021-01-28 11:08:43 +01:00
Michael Eischer
0caad1e890
copy: parallelize tree walk
2021-01-28 11:08:43 +01:00
Alexander Neumann
a4689eb3b9
Merge pull request #3199 from MichaelEischer/non-interactive-counter
...
Don't print progress on non-interactive terminals
2021-01-28 10:53:38 +01:00
Alexander Weiss
d7dc19a496
prune: Always repack packs containing tree blobs
2021-01-15 16:42:04 +01:00
Michael Eischer
f3442ce8a5
Test that WriteTo of a backend's Load remains accessible
2021-01-03 22:23:53 +01:00
Michael Eischer
97388b3504
Properly check that --quiet and --verbose are not combined
...
If --verbose is specified once, then globalOptions.Verbose == 1.
Previously --quiet --verbose would silently ignore the --verbose flag.
2020-12-30 21:24:18 +01:00
MichaelEischer
e1efc193e1
Merge pull request #3139 from aawsome/prune-healing
...
prune: Add healing of repository in some situations
2020-12-29 22:17:27 +01:00
Alexander Weiss
f0113139ea
prune: Correct error message
2020-12-29 20:20:05 +01:00
Alexander Weiss
f6df94a50e
prune: Add self-healing
...
Allow prune to heal situations where blobs in the index are missing or
the corresponding packfiles are damaged if those blobs are not needed.
2020-12-29 20:20:05 +01:00
MichaelEischer
31e56f1ad5
Merge pull request #3197 from SkYNewZ/fix/3183
...
Fix tag handling for multiple tag lists
2020-12-29 18:44:38 +01:00
greatroar
dec5008369
Warn when unlock fails instead of returning an error
...
Only one caller was checking the error.
2020-12-29 17:48:20 +01:00
Quentin Lemaire
25ecf9eafb
fix(cmd_tag): Use restic.TagLists
2020-12-29 17:12:46 +01:00
Quentin Lemaire
e88f3fb80c
fix(cmd_backup): Use restic.TagLists
2020-12-29 17:12:46 +01:00
Michael Eischer
05a987b07c
Support values less than 1 for RESTIC_PROGRESS_FPS
...
For example set the variable to 0.016666 to print the progress once per
minute.
2020-12-29 16:32:18 +01:00
Michael Eischer
85fe5feadb
Unify progress report frequency calculation
2020-12-29 16:03:43 +01:00
Michael Eischer
969141b5e9
Honor RESTIC_PROGRESS_FPS env variable on non-interactive terminals
...
This makes it possible to use the environment variable to also get
regular progress updates on non-interactive terminals.
2020-12-29 16:03:43 +01:00
Michael Eischer
c2ef049f1b
ui/progress: don't print progress on non-interactive terminals
...
This reverts to the old behavior of not printing progress updates on
non-interactive terminals. It was accidentally changed in #3058 .
2020-12-29 16:03:43 +01:00
greatroar
18531e3d6f
Portability fixes to internal/restic
...
syscall.Mknod is not available on AIX.
2020-12-29 01:35:01 +01:00
DRON-666
332b1896d1
Some options
fixes
...
Add tests for bool type.
Fix subtle bug in TestOptionsApplyInvalid.
Fix options list formatting.
2020-12-23 23:26:04 +03:00
Alexander Weiss
d8d2cc6dd9
Simplify rebuild-index code
2020-12-22 23:01:58 +01:00
Alexander Weiss
68b74e359e
Count packs directly in RebuildIndexFiles
2020-12-22 23:01:58 +01:00
Michael Eischer
ccc84af73d
debug/list: parallelize index loading
2020-12-22 22:36:18 +01:00
Alexander Neumann
edf89e1c74
forget: Enable --dry-run together with --prune
2020-12-22 20:58:02 +01:00
Michael Eischer
cfea79d0c5
prune: don't print stacktrace on console
2020-12-19 14:28:48 +01:00
DRON-666
2ca76afc2b
dump: Add new option --archive
2020-12-19 01:09:47 +03:00
DRON-666
0256f95994
dump: Split tar and walk logic
2020-12-19 01:09:47 +03:00
Alexander Neumann
0d5b764f90
Merge pull request #3130 from aawsome/snapshots-parallel
...
Make loading snapshots parallel
2020-12-06 21:08:18 +01:00
Alexander Weiss
e329623771
Remove LoadAllSnapshots
2020-12-06 05:22:27 +01:00
Alexander Weiss
5b9ee56335
Add ForAllSnapshots
2020-12-06 05:04:21 +01:00
Alexander Weiss
83c8a9b058
Bugfix: packSizeFromList should save size from List()
2020-12-05 20:58:36 +01:00
Michael Eischer
f72f6c9c80
Fix debug build
2020-11-29 18:44:36 +01:00
MichaelEischer
52b98f7f95
Merge pull request #3017 from greatroar/files-from0
...
Add backup options --files-from-verbatim and --files-from-raw
2020-11-29 18:15:21 +01:00
Alexander Neumann
04d856e601
Merge pull request #3136 from restic/rawtaz-copy-doc
...
doc: Emphasize double transfer and duplication in copy command
2020-11-29 13:57:10 +01:00
Alexander Neumann
a7b49c4889
Merge pull request #3119 from restic/keep-mountpoints
...
Keep mountpoints as empty directories for --one-file-system
2020-11-29 11:22:12 +01:00
Leo R. Lundgren
f70b10d0ee
doc: Emphasize double transfer and duplication in copy command
2020-11-28 19:49:34 +01:00
greatroar
55bf76ba0c
backup: Add --files-from-{verbatim,raw} options
2020-11-28 18:22:31 +01:00
Alexander Neumann
82ae942965
backup: Keep mountpoints for --one-file-system
...
When a file system is mounted at a directory, lstat() returns attributes
of the root node of the mounted file system, including the device ID of
the other file system. The previous code used when --one-file-system is
specified excluded the directory itself because of that.
This commit changes the code so that mountpoints are kept as empty
directories, its attributes set to the root note of the mounted file
system. The behavior mimics `tar`, which does the same.
2020-11-28 17:00:31 +01:00
Alexander Neumann
f576d3d826
Add tests
2020-11-28 17:00:31 +01:00
Alexander Neumann
037f0a4c91
Refactor device ID checking
2020-11-28 17:00:31 +01:00
Alexander Neumann
a3105799c9
Print warning when unlocking the repo fails
2020-11-28 11:31:25 +01:00
Kyle Brennan
666768cd17
cat: Respect --no-lock flag
2020-11-28 11:30:02 +01:00
Alexander Weiss
aa7a5f19c2
Use BlobHandle in index methods
2020-11-22 20:41:12 +01:00
Alexander Weiss
92bd448691
Make BlobHandle substruct of Blob
2020-11-22 20:41:10 +01:00
Alexander Weiss
67c938f232
Use PackSize in prune
2020-11-21 22:13:54 +01:00
Alexander Weiss
4960b841e6
Use PackSize in rebuild-index
2020-11-21 22:13:54 +01:00
Alexander Neumann
110a32a08b
Merge pull request #3113 from aawsome/fix-prune-stats-duplicates
...
Fix statistics in prune for duplicates
2020-11-19 20:34:11 +01:00
Alexander Weiss
cb5ec7ea6b
Fix statistics in prune for duplicates
...
Note that this fix only solves the statistics problem, if
all duplicates are marked for repacking.
If not all duplicates are marked for repacking, we lack the
information which
The situation that not all duplicates are marked for repacking can occur
when using the `max-repack-size` option
2020-11-18 22:30:22 +01:00
Michael Eischer
625410f003
Limit progress bar updates to once per second on non-terminal outputs
...
The code accidentally checked whether stdin is a terminal instead of
stdout, the former is not relevant here as the output is printed on
stdout.
2020-11-18 22:12:07 +01:00
Alexander Neumann
d857fb6e59
Allow using --repository-file in init
2020-11-17 20:43:46 +01:00
Michael Eischer
1f43cac12d
check: Only track data blobs when unused blobs should be reported
...
This improves the memory usage of check a lot as it now only has to
track tree blobs when run using the default parameters.
2020-11-15 18:43:07 +01:00
Michael Eischer
3500f9490c
check: Simplify blob status tracking
...
UnusedBlobs now directly reads the list of existing blobs from the
repository index. This removes the need for the blobStatusExists flag,
which in turn allows converting the blobRefs map into a BlobSet.
2020-11-15 18:42:42 +01:00
Michael Eischer
caac38ed27
check: Tweak subset percentage over 100% error message
2020-11-15 18:13:50 +01:00
fgma
8f9cea8cc0
Check data subset: check random percentage subset
2020-11-15 18:13:50 +01:00
Alexander Neumann
3c0c0c132b
Merge pull request #3006 from aawsome/new-rebuild-index
...
Reimplement rebuild-index and remove /internal/index
2020-11-15 17:48:43 +01:00
kitone
3c03b35212
Fix #1681 should not try to create the mount point if it doesn't exist, rather return an error
2020-11-15 17:09:30 +01:00
Alexander Weiss
9607cad267
Remove internal/index
2020-11-15 07:05:09 +01:00
Alexander Weiss
30b6a0878a
Reimplement rebuild-index
2020-11-15 07:05:09 +01:00
Alexander Weiss
1ec628ddf5
Add extraObsolete to MasterIndex.Save
2020-11-15 07:05:09 +01:00
MichaelEischer
3601a9b6cd
Merge pull request #2690 from SkYNewZ/master
...
Fix #2688 : Handle comma-separated list tags when using backup command
2020-11-14 23:19:42 +01:00
Michael Eischer
fdec8051ab
tags: Tweak description of --add/set/remove options
2020-11-14 22:55:30 +01:00
MichaelEischer
333c5a19d4
Merge pull request #3082 from aawsome/check-sizes
...
Check: check sizes of packs from index and packheader
2020-11-14 22:37:42 +01:00
Quentin Lemaire
a8ad6b9a4b
fix(tags): Change tags list type according to restic.TagList
2020-11-14 16:24:58 +00:00
Quentin Lemaire
b0882b3f3c
fix(snapshots): Update help message to better match the 'forget' command one
2020-11-14 15:48:57 +00:00
Quentin Lemaire
ae441d3134
fix(backup): Switch tags cobra type to handle comma-separated list
2020-11-14 15:48:56 +00:00
Alexander Weiss
80dcfca191
check: Check sizes computed from index and pack header
2020-11-14 00:42:49 +01:00
Alexander Neumann
59ea5a4208
diff: Correctly count top-level blobs
2020-11-13 21:11:21 +01:00
greatroar
63e32c44c0
Improve error reporting from restic diff
...
Instead of a stacktrace, restic diff 111 222 now reports:
Fatal: no matching ID found for prefix "111"
2020-11-11 16:40:40 +01:00
greatroar
21b787a4d1
Stop Counters where they're constructed and started
2020-11-09 13:03:31 +01:00
greatroar
ddca699cd2
Replace restic.Progress with new progress.Counter
...
This fixes two race conditions while cleaning up the code.
2020-11-09 12:12:35 +01:00
Alexander Neumann
a8d21b5dcf
prune: Warn if no cache is present
2020-11-08 20:25:35 +01:00
Alexander Neumann
823d0afd6e
backup: Always print parent snapshot info
2020-11-08 20:25:35 +01:00
Alexander Neumann
47277c4b4c
Add comments, clarify computation
2020-11-06 20:23:30 +01:00
Alexander Weiss
d2e53730d6
Add test that repo.List is only called once
2020-11-06 20:23:30 +01:00
Alexander Weiss
fd33030556
Use in-memory index to rebuild index in prune
2020-11-06 20:23:30 +01:00
Alexander Neumann
5144141321
Merge pull request #2718 from aawsome/new-cleanup-command
...
Reimplementation of prune
2020-11-05 10:12:19 +01:00
Alexander Neumann
d35d279455
Set development version for 0.11.0
2020-11-05 09:41:40 +01:00
Alexander Neumann
b7e1ece1e0
Add version for 0.11.0
2020-11-05 09:41:35 +01:00
fgma
916b2d303b
vss: fix potential crash (not reachable in restic) ( #3045 )
...
HasSufficientPrivilegesForVSS() now returns an error
2020-11-04 22:14:18 +01:00
Alexander Neumann
1ca60bccfb
Refactor condition for MaxRepackBytes
...
Don't depend on the string (opts.MaxRepackSize) for the condition,
instead check if there's a (positive) limit configured.
2020-11-03 16:42:21 +01:00
Alexander Neumann
7f86eb4ec0
Move helper function
2020-11-03 16:42:21 +01:00
Alexander Neumann
c1a3de4a6e
Refactor max-unused calculation, add unlimited
option
...
Add a callback to the PruneOptions struct which calculates the number of
bytes allowed to be unused after prune is done. This way, the logic is
closer to the option parsing code.
Also, add an explicit option `unlimited` for the use case when storage
does not matter but bandwidth and time do. Internally, this sets the
maximum number of unused bytes to MaxUint64.
Rework the documentation slightly so that no more "packs" are
mentioned and it talks about "files" instead.
Make it clear in the documentation that the percentage given to
`--max-unused` is relative to the whole repository size after pruning is
done. If specified, it must be below 100%, otherwise the repository
would contain 100% of unused data, which is pointless.
I had a hard time coming up with the correct formula to calculate the
maximum number of unused bytes based on the number of used bytes. For a
fraction `p` (0 ≤ p < 1), a repo with `u` bytes used, and the number of
unused bytes `x` the following holds:
x ≤ p * (u+x)
⇔ x ≤ p*u + p*x
⇔ x - p*x ≤ p*u
⇔ x * (1-p) ≤ p*u
⇔ x ≤ p/(1-p) * u
2020-11-03 16:42:21 +01:00
Alexander Neumann
f8c4dd7b1a
Split packe rewrite logic into two case branches
...
The comma is too sublte, let's split this into two separate branches.
2020-11-03 16:42:21 +01:00
Alexander Neumann
a5b80452fe
Add comment that usedBlobs is modified
2020-11-03 16:42:21 +01:00
Alexander Neumann
aff1e220f5
Split struct members, add comments
2020-11-03 16:42:21 +01:00
Alexander Neumann
095155d9ce
Remove RepackSmall
2020-11-03 16:42:21 +01:00
Alexander Weiss
b2f5381737
Make realistic forget --prune --dryrun
2020-11-03 16:42:21 +01:00
Alexander Weiss
7f9a0a5907
Reimplementation of prune
2020-11-03 16:42:21 +01:00
Alexander Weiss
3b591ed987
Add Verboseff
2020-11-03 16:42:21 +01:00
Alexander Neumann
21ba15577e
Return an error for invalid sizes
2020-11-03 11:53:49 +01:00
Alexander Neumann
6509c207f4
Merge pull request #2997 from MichaelEischer/faster-excludes
...
Speedup exclude/include checking
2020-11-02 11:07:21 +01:00
Alexander Neumann
445b845267
Merge pull request #2978 from MichaelEischer/warn-tree-error
...
Warn if backup failed to read tree blob
2020-11-02 10:14:12 +01:00
Alexander Neumann
3ff37215df
Merge pull request #2935 from MichaelEischer/upgrade-minio
...
Upgrade minio SDK to version 7
2020-11-02 09:09:10 +01:00
MichaelEischer
31b8d7a639
Merge pull request #2274 from fgma/master
...
Support for Volume Shadow Copy Service (VSS) on windows
2020-10-24 15:43:34 +02:00
fgma
5695f9ebd2
vss: Implement VSS support for Windows
...
The VSS support works for 32 and 64-bit windows, this includes a check that
the restic version matches the OS architecture as required by VSS. The backup
operation will fail the user has not sufficient permissions to use VSS.
Snapshotting volumes also covers mountpoints but skips UNC paths.
2020-10-24 11:35:57 +02:00
greatroar
3807d13bdc
restic mount: don't umount unless we actually tried to mount
...
Previously, failure to open a repository would result in an "unable to
umount" message.
2020-10-21 12:15:05 +02:00
Alexander Neumann
56883817d8
Merge pull request #2990 from MichaelEischer/fix-goreport-warnings
...
Fix some goreport warnings
2020-10-12 20:44:56 +02:00
Michael Eischer
e638b46a13
Embed context into ReaderAt
...
The io.Reader interface does not support contexts, such that it is
necessary to embed the context into the backendReaderAt struct. This has
the problem that a reader might suddenly stop working when it's
contained context is canceled. However, this is now problem here as the
reader instances never escape the calling function.
2020-10-09 22:39:07 +02:00
Michael Eischer
2964d2ad15
Skip unlocking for nil locks
...
Now that lockRepo receives a context, it is possible that it is canceled
before a lock was created. Thus `unlockRepo` must be able to handle this
case.
2020-10-09 22:39:06 +02:00
Michael Eischer
0c9efa9c2a
Pass context to lockRepo
2020-10-09 22:39:06 +02:00
Michael Eischer
37a5e2d681
rest: use global context on repository creation
2020-10-09 22:39:06 +02:00
Michael Eischer
dc2e664209
integration_fuse_test: use global context
...
No need to use the TODO context.
2020-10-09 22:37:56 +02:00
Michael Eischer
603bb0e309
restore: Use proper context while loading snapshot
2020-10-09 22:37:56 +02:00
Michael Eischer
27456f6545
debug: use proper context
...
This allows the debug commands to be properly interrupted.
2020-10-09 22:37:56 +02:00
Michael Eischer
c458e114d4
pass context to Find / FindSnapshot
...
This allows proper interruption of restic while it searches for
snapshots or key files.
2020-10-09 22:37:56 +02:00
Michael Eischer
45e9a55c62
Wire context into backend layout detection
2020-10-09 22:37:24 +02:00
Michael Eischer
307a6ba3a3
Upgrade minio sdk to v7
...
This changes are primarily straightforward modifications to pass the
parameters in the now expected way.
2020-10-09 22:37:24 +02:00
Michael Eischer
50da20d93d
Warn if backup failed to read tree blob
2020-10-09 22:36:27 +02:00
Alexander Neumann
30cb553c8d
Merge pull request #2932 from MichaelEischer/proper-rclone-create
...
Call rclone.Create to create a new repository for the rclone backend
2020-10-09 21:29:15 +02:00
Michael Eischer
bcc3bddcf4
filter: only check whether a child path could match when necessary
...
When checking excludes there is no need to test whether a child path
could also match the pattern, as it is by definition excluded.
Previously childMayMatch was calculated but then discarded. For simple
absolute paths this can account for half the time spent for checking
pattern matches.
name old time/op new time/op delta
FilterPatterns/Relative-4 23.3ms ± 9% 21.7ms ± 6% -6.68% (p=0.004 n=10+10)
FilterPatterns/Absolute-4 13.9ms ± 7% 10.0ms ± 5% -27.61% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 51.4ms ± 7% 47.0ms ± 7% -8.51% (p=0.001 n=9+9)
FilterPatterns/ManyNoMatch-4 551ms ± 9% 190ms ± 1% -65.41% (p=0.000 n=10+8)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.665 n=10+9)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.480 n=9+10)
FilterPatterns/Wildcard-4 14.3MB ± 0% 14.3MB ± 0% ~ (p=0.431 n=9+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:52 +02:00
Michael Eischer
375c2a56de
filter: Parse filter patterns only once
...
name old time/op new time/op delta
FilterPatterns/Relative-4 30.3ms ±10% 23.6ms ±20% -22.12% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 49.0ms ± 3% 32.3ms ± 8% -33.94% (p=0.000 n=8+10)
FilterPatterns/Wildcard-4 345ms ± 9% 334ms ±17% ~ (p=0.315 n=10+10)
FilterPatterns/ManyNoMatch-4 3.93s ± 2% 0.71s ± 7% -81.98% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 4.63MB ± 0% 3.57MB ± 0% -22.98% (p=0.000 n=9+9)
FilterPatterns/Absolute-4 8.54MB ± 0% 3.57MB ± 0% -58.20% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 146MB ± 0% 141MB ± 0% -2.93% (p=0.000 n=9+9)
FilterPatterns/ManyNoMatch-4 907MB ± 0% 4MB ± 0% -99.61% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 66.6k ± 0% 22.2k ± 0% -66.67% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 88.7k ± 0% 22.2k ± 0% -75.00% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 1.70M ± 0% 1.63M ± 0% -3.92% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 4.46M ± 0% 0.02M ± 0% -99.50% (p=0.000 n=10+10)
2020-10-07 20:47:27 +02:00
MichaelEischer
0ae02f3030
Merge pull request #2844 from aawsome/prune-integration-tests
...
Add prune integration tests for many edge cases
2020-10-06 23:16:41 +02:00
Lorenz Bausch
af4100e07d
Replace repo/repos with repository/repositories in --no-lock help text
2020-10-06 22:49:48 +02:00
Alexander Weiss
6822a58413
Add prune integration tests for many edge cases
2020-10-06 20:20:05 +02:00
Michael Eischer
1579d2a8ec
Remove some unused assignments
2020-10-06 14:55:13 +02:00
Michael Eischer
eba5dd831f
Fix typos reported by misspell
2020-10-06 14:55:13 +02:00
Leo R. Lundgren
e105a3f391
cache/check: Clarify error message when given arguments
2020-10-06 00:08:59 +02:00
rawtaz
5d8cfff3f2
Merge pull request #2988 from greatroar/snapshots-json
...
Regression test for #2979
2020-10-05 21:02:52 +02:00
MichaelEischer
1ab4c710e1
Merge pull request #2987 from greatroar/no-password-for-help
...
Don't read password for generate, help or self-update
2020-10-05 20:16:19 +02:00
greatroar
feedf0ebce
Don't read password for generate, help or self-update
...
Fixes #2951 .
2020-10-05 19:54:07 +02:00
MichaelEischer
1a490acd67
Merge pull request #2982 from greatroar/archiver-error-handling
...
Check error in archiver before calling Select
2020-10-05 19:19:47 +02:00
greatroar
6bee0aafc2
Regression test for #2979
2020-10-05 15:08:09 +02:00
Leo R. Lundgren
137d20a06a
snapshots: Make --json output [] instead of null when no snapshots
2020-10-05 13:14:10 +02:00
greatroar
c4e2203e45
Check error in archiver before calling Select
...
The archiver first called the Select function for a path before checking
whether the Lstat on that path actually worked. The RejectFuncs in
exclude.go worked around this by checking whether they received a nil
os.FileInfo. Checking first is more obvious and requires less code.
2020-10-05 11:11:04 +02:00
rawtaz
bbeb439f41
Merge pull request #2937 from andreaso/self-update-output-path-fix
...
Don't require `self-update --output` placeholder file
2020-10-03 13:35:20 +02:00
Andreas Olsson
ce14df303b
Don't require self-update --output
placeholder file
...
This removes the requirement on `restic self-update --output` to point
to a path of an existing file, to overwrite. In case the specified
path does exist we still want to verify that it's a regular file,
rather than a directory or a device, which gets overwritten.
We also want to verify that a path to a new file exists within an
existing directory. The alternative being running into that issue
after the actual download, etc has completed.
While at it I also replace `errors.Errorf` with the more appropriately
verbose `errors.Fatalf`.
Resolves #2491
2020-10-03 07:02:23 +02:00
MichaelEischer
3c6671b18b
Merge pull request #2973 from restic/rawtaz-repo2
...
Correct data type for --repo2 option
2020-10-02 22:22:41 +02:00
rawtaz
0cce6dc31c
Correct data type for --repo2 option
2020-10-02 21:10:52 +02:00
rawtaz
6253ff0187
Clarify max verbose level in option description
...
Clarifies that the max verbosity level is 3, in the description of the --verbose option.
2020-10-02 20:25:34 +02:00
jtagcat
a687261804
--help: --compact: clarify description
2020-10-02 16:55:56 +03:00
Michael Eischer
61035d68bc
Add test for --repository-file
2020-10-01 00:50:27 +02:00
Andreas Oberritter
97f7855de3
Add new option --repository-file (default: $RESTIC_REPOSITORY_FILE)
...
As an alternative to -r, this allows to read the repository URL
from a file in order to prevent certain types of information leaks,
especially for URLs containing credentials.
Fixes #1458 , fixes #2900 .
2020-10-01 00:50:26 +02:00
rawtaz
34ea960559
Merge pull request #2966 from MichaelEischer/recover-help-typo
...
recover: Fix typo in the command help
2020-09-30 18:16:26 +02:00
Michael Eischer
2a79c1a44d
recover: Fix typo in the command help
...
Reported-by: Steve Divskinsy <stevesbrain@users.noreply.github.com>
2020-09-30 17:44:34 +02:00
Michael Eischer
aea9f7d286
clarify verbose password prompt
2020-09-30 17:25:54 +02:00
plumbeo
009bd907f2
Don’t print too many messages when deleting files
...
Print detailed debug messages on file deletions only when --verbose=2 is used
2020-09-27 14:24:04 +02:00
MichaelEischer
14b312f00d
Merge pull request #2658 from creativeprojects/issue-2241
...
Don't echo authentication passwords (rest backend)
2020-09-22 22:17:53 +02:00
Fred
206cadfab4
Hide password from repository URLs
2020-09-22 22:00:51 +02:00
MichaelEischer
4875f7b659
Merge pull request #2614 from greatroar/simplify-fs
...
Simplify internal/fs
2020-09-21 22:05:10 +02:00
yoshiera
3cf29a777d
Fix nil check in rejectBySize
2020-09-21 19:20:24 +08:00
Michael Eischer
9333f707fa
init: use Create method for rclone backend
...
This properly issues the initial repository creation command
Fixes #1896
2020-09-19 19:11:43 +02:00
Alexander Neumann
429f97b887
Set development version for 0.10.0
2020-09-19 17:38:47 +02:00
Alexander Neumann
40832b2927
Add version for 0.10.0
2020-09-19 17:38:26 +02:00
Michael Eischer
f003410402
init: Add --copy-chunker-params
option
...
This allows creating multiple repositories with identical chunker
parameters which is required for working deduplication when copying
snapshots between different repositories.
2020-09-19 16:53:05 +02:00
Michael Eischer
655430550b
Extract parameters for second repository from copy command
2020-09-19 16:07:55 +02:00
Leo R. Lundgren
311ad2d2d0
Improve wording for --password-file and related options
2020-09-19 15:47:32 +02:00
Leo R. Lundgren
ffe6dce7e7
Improve wording for --password-command and --password-command2 options
2020-09-19 14:23:25 +02:00
yoshiera
ac4b8c98ac
Support excluding files by size
2020-09-19 19:52:09 +08:00
greatroar
9abef3bf1a
Move internal/fs.TestChdir to internal/test.Chdir
2020-09-17 10:43:33 +02:00