KurlesHS/restic
1
0
Fork 0
forked from TrueCloudLab/restic
Code Pull requests Activity
5326 commits 10 branches 49 tags 79 MiB
Commit graph

992 commits

Author SHA1 Message Date
Alexander Neumann
75f53955ee errcheck: Add error checks 
Most added checks are straight forward.
 2021-01-30 20:02:37 +01:00
Michael Eischer
a53778cd83 rest: handle dropped error in save operation 2021-01-30 19:25:04 +01:00
Michael Eischer
8a486eafed gs: Don't drop error when finishing upload 
The error returned when finishing the upload of an object was dropped.
This could cause silent upload failures and thus data loss in certain
cases. When a MD5 hash for the uploaded blob is specified, a wrong
hash/damaged upload would return its error via the Close() whose error
was dropped.
 2021-01-30 13:31:32 +01:00
Alexander Neumann
cdd704920d azure: Pass data length to Azure libray 
The azureAdapter was used directly without a pointer, but the Len()
method was only defined with a pointer receiver (which means Len() is
not present on a azureAdapter{}, only on a pointer to it).
 2021-01-29 21:08:41 +01:00
Alexander Neumann
bbdf18c4a2
Merge pull request #3176 from MichaelEischer/backend-content-length 
Pass upload size to backends and sanity check it
 2021-01-29 20:33:44 +01:00
Michael Eischer
1f583b3d8e backend: test that incomplete uploads fail 2021-01-29 13:51:53 +01:00
Michael Eischer
c73316a111 backends: add sanity check for the uploaded file size 
Bugs in the error handling while uploading a file to the backend could
cause incomplete files, e.g. https://github.com/golang/go/issues/42400
which could affect the local backend.

Proactively add sanity checks which will treat an upload as failed if
the reported upload size does not match the actual file size.
 2021-01-29 13:51:51 +01:00
Michael Eischer
4526d5d197 swift: explicitly pass upload size to library 
This allows properly setting the content-length which could help the
server-side to detect incomplete uploads.
 2021-01-29 13:50:46 +01:00
Michael Eischer
dca9b6f5db azure: explicitly pass upload size 
Previously the fallback from the azure library was to read the whole
blob into memory and use that to determine the upload size.
 2021-01-29 13:50:46 +01:00
greatroar
6bd8a2faaa backup: Add --ignore-ctime option and document change detection 2021-01-28 23:42:10 +01:00
MichaelEischer
43cb26010a
Merge pull request #3242 from greatroar/fprintln 
internal/ui/termstatus: Use Fprintln to get a newline
 2021-01-28 20:34:06 +01:00
Alexander Neumann
4d40c70214
Merge pull request #3211 from MichaelEischer/sftp-speedup 
Speed-up caching/pack download via SFTP
 2021-01-28 14:16:55 +01:00
Alexander Neumann
6aa7e9f9c6
Merge pull request #3174 from MichaelEischer/parallelize-lock-loading 
Parallelize lock file loading
 2021-01-28 13:52:12 +01:00
Alexander Neumann
bdfedf1f5b
Merge pull request #3173 from MichaelEischer/unify-index-loading 
Unify index loading
 2021-01-28 13:50:42 +01:00
greatroar
b9cfe6f68a internal/ui/termstatus: Use Fprintln to get a newline 2021-01-28 13:30:10 +01:00
Michael Eischer
68608a89ad restic: add comment about StreamTrees shutdown 2021-01-28 11:10:50 +01:00
Michael Eischer
ddb7697d29 restic: Test progress reporting of StreamTrees 2021-01-28 11:10:50 +01:00
Michael Eischer
313ad0e32f progress/counter: Fix test for final report call 2021-01-28 11:10:50 +01:00
Michael Eischer
e2b0072441 check: add progress bar to the tree structure check 2021-01-28 11:10:50 +01:00
Michael Eischer
505f8a2229 progress/counter: Support updating the progress bar maximum 2021-01-28 11:10:47 +01:00
Michael Eischer
eda8c67616 restic: let FindUsedBlobs handle multiple snapshots at once 2021-01-28 11:08:43 +01:00
Michael Eischer
258ce0c1e5 parallel: report progress for StreamTrees 
This assigns an id to each tree root and then keeps track of how many
tree loads (i.e. trees referenced for the first time) are pending per
tree root. Once a tree root and its subtrees were fully processed there
are no more pending tree loads and the tree root is reported as
processed.
 2021-01-28 11:08:43 +01:00
Michael Eischer
f2a1b125cb restic: Actually parallelize FindUsedBlobs 2021-01-28 11:08:43 +01:00
Michael Eischer
6e03f80ca2 check: Split the parallelized tree loader into a reusable component 
The actual code change is minimal
 2021-01-28 11:08:43 +01:00
Michael Eischer
1d7bb01a6b check: Cleanup tree loading and switch to use errgroup 
The helper methods are now wired up in the Structure method.
 2021-01-28 11:08:43 +01:00
Alexander Neumann
a4689eb3b9
Merge pull request #3199 from MichaelEischer/non-interactive-counter 
Don't print progress on non-interactive terminals
 2021-01-28 10:53:38 +01:00
Alexander Neumann
c5a66e9181 ui: Simlify channel receive 2021-01-28 10:42:02 +01:00
Michael Eischer
678e75e1c2 sftp: enforce use of optimized upload method 
ReadFrom was already used by Save before, this just ensures that this
won't accidentally change in the future.
 2021-01-03 22:23:53 +01:00
Michael Eischer
6b5b29dbee limiter: add unit tests 2021-01-03 22:23:53 +01:00
Michael Eischer
f35f2c48cd limiter: support WriteTo in LimitBackend for read rate limiting 2021-01-03 22:23:53 +01:00
Michael Eischer
bcb852a8d0 hashing: support WriteTo in the reader 2021-01-03 22:23:53 +01:00
Michael Eischer
69d5b4c36b restorer: lower-case variable name 2021-01-03 13:55:59 +01:00
Alexander Weiss
34a33565c8 Fix loadBlob in filerestorer 2021-01-01 08:06:04 +01:00
Alexander Weiss
7409225fa8 Add filerestorer test where only parts of pack are used 2021-01-01 07:24:46 +01:00
Alexander Weiss
07b3f65a6f filesrestorer: Re-use buffer 2021-01-01 07:24:46 +01:00
Alexander Weiss
3e0acf1395 restore: Don't save (part of) pack in memory 2021-01-01 07:24:46 +01:00
Alexander Neumann
8b84c96d9d
Merge pull request #3204 from MichaelEischer/archiver-tomb-race 
archiver: fix race condition during worker startup
 2020-12-30 20:04:20 +01:00
Michael Eischer
debc4a3a99 archiver: fix race condition during worker startup 
When the tomb is created with a canceled context, then the workers
started via `t.Go` exist nearly immediately. Once for the first time all
started goroutines have been stopped, it is not allowed to issue further
calls to `t.Go`. This is a problem when the started goroutines exit
immediately, as for example the first goroutine might already have
stopped before starting the second one, which is not allowed as once the
first goroutines has stopped no goroutines were running.

To fix this race condition the startup and main task of the archiver now
also run within a `t.Go` function. This also allows unifying the error
handling as it is no longer necessary to distinguish between errors
returned by the workers or the saveTree processing. The tomb now just
returns the first error encountered, which should also be the most
descriptive one.
 2020-12-30 17:31:22 +01:00
MichaelEischer
31e56f1ad5
Merge pull request #3197 from SkYNewZ/fix/3183 
Fix tag handling for multiple tag lists
 2020-12-29 18:44:38 +01:00
Alexander Neumann
b2efa0af39
Merge pull request #3164 from MichaelEischer/improve-context-cancel 
Improve context cancel handling in archiver and backends
 2020-12-29 17:03:42 +01:00
Michael Eischer
cff4955a48 ui: remove dead struct member 2020-12-29 16:32:18 +01:00
Michael Eischer
92da5168e1 ui: force backup progress update on signal 2020-12-29 16:32:18 +01:00
Michael Eischer
34afc93ddc ui/progress: extract signal handling into own package 2020-12-29 16:32:18 +01:00
Michael Eischer
023eea6463 ui: don't shorten non-interactive progress output 2020-12-29 16:32:18 +01:00
Michael Eischer
684600cf42 ui: update status for the backup command on non-interactive terminals 
Allow the backup command to print status on non-interactive terminals.
The output is disabled by setting a MinUpdatePause == 0.
 2020-12-29 16:03:43 +01:00
Michael Eischer
13ce981794 ui: cleanup backup status shutdown 2020-12-29 16:03:43 +01:00
Michael Eischer
c2ef049f1b ui/progress: don't print progress on non-interactive terminals 
This reverts to the old behavior of not printing progress updates on
non-interactive terminals. It was accidentally changed in #3058.
 2020-12-29 16:03:43 +01:00
Quentin Lemaire
334d8ce724
feat(tags): Create Flatten() method 2020-12-29 10:59:46 +01:00
greatroar
3b09ae9074 AIX port 2020-12-29 01:35:01 +01:00
greatroar
18531e3d6f Portability fixes to internal/restic 
syscall.Mknod is not available on AIX.
 2020-12-29 01:35:01 +01:00