Commit graph

8056 commits

Author SHA1 Message Date
Michael Eischer
4f45668b7c repository: rework and extend LoadRaw tests 2024-05-18 21:28:54 +02:00
Michael Eischer
ac805d6838 cache: cleanup debug logs 2024-05-18 21:28:54 +02:00
Michael Eischer
5214af88e2 cache: test forget behavior 2024-05-18 21:28:54 +02:00
Michael Eischer
3ff063e913 check: verify pack a second time if broken 2024-05-18 21:28:54 +02:00
Michael Eischer
385cee09dc repository: fix caching of tree packs in LoadBlobsFromPack 2024-05-18 21:28:54 +02:00
Michael Eischer
e734746f75 cache: forget cached file at most once
This is inspired by the circuit breaker pattern used for distributed
systems. If too many requests fails, then it is better to immediately
fail new requests for a limited time to give the backend time to
recover.

By only forgetting a file in the cache at most once, we can ensure that
a broken file is only retrieved once again from the backend. If the file
stored there is broken, previously it would be cached and deleted
continuously. Now, it is retrieved only once again, all later requests
just use the cached copy and either succeed or fail immediately.
2024-05-18 21:28:54 +02:00
Michael Eischer
97a307df1a cache: Always use cached file if it exists
A file is always cached whole. Thus, any out of bounds access will also
fail when directed at the backend. To handle case in which the cached
file is broken, then caller must call Cache.Forget(h) for the file in
question.
2024-05-18 21:28:54 +02:00
Michael Eischer
8cce06d915 repair packs: drop experimental warning
This warning should already have been removed once the feature flag was
dropped.
2024-05-18 21:28:54 +02:00
Michael Eischer
433a6aad29 repository: remove redundant blob loading fallback from RepairPacks
LoadBlobsFromPack already implements the same fallback behavior.
2024-05-18 21:28:54 +02:00
Michael Eischer
e401af07b2 check: fix error message formatting 2024-05-18 21:28:54 +02:00
Michael Eischer
7017adb7e9 repository: retry failed ListPack once 2024-05-18 21:28:54 +02:00
Michael Eischer
e33ce7f408 repository: retry failed LoadBlob once 2024-05-18 21:28:54 +02:00
Michael Eischer
2ace242f36 repository: make reloading broken files explicit 2024-05-18 21:28:54 +02:00
Michael Eischer
e9390352a7 cache: code cleanups 2024-05-18 21:26:00 +02:00
Michael Eischer
503c8140b1 repository: unify blob decoding code 2024-05-18 21:26:00 +02:00
Michael Eischer
6563f1d2ca repository: remove redundant debug log 2024-05-18 21:26:00 +02:00
Michael Eischer
021fb49559 repository: Implement repository.LoadUnpacked using LoadRaw
Both functions were using a similar implementation.
2024-05-18 21:26:00 +02:00
Michael Eischer
779c8d3527 debug/repair packs/upgrade repo v2: use repository.LoadRaw
This replaces calling the low-level backend.Load() method.
2024-05-18 21:26:00 +02:00
Michael Eischer
1d6d3656b0 repository: move backend.LoadAll to repository.LoadRaw
LoadRaw also includes improved context cancellation handling similar to the
implementation in repository.LoadUnpacked.

The removed cache backend test will be added again later on.
2024-05-18 21:26:00 +02:00
Michael Eischer
47232bf8b0 backend: move LimitReadCloser to util package
The helper is only intended for usage by backend implementations.
2024-05-18 21:26:00 +02:00
Michael Eischer
dcd151147c
Merge pull request #4803 from restic/permanent-retry-failure
Do not retry permanent backend failures
2024-05-18 20:07:06 +02:00
Michael Eischer
53d15bcd1b retry: add circuit breaker to load method
If a file exhausts its retry attempts, then it is likely not accessible
the next time. Thus, immediately fail all load calls for that file to
avoid useless retries.
2024-05-18 19:59:26 +02:00
Michael Eischer
394c8ca3ed rest/rclone/s3/sftp/swift: move short file detection behind feature gate
These backends tend to use a large variety of server implementations.
Some of those implementations might prove problematic with the new
checks.
2024-05-18 19:59:26 +02:00
Michael Eischer
6328b7e1f5 replace "too small" with "too short" in error messages 2024-05-18 19:59:26 +02:00
Michael Eischer
53561474d9 update changelog with persistent backend error handling 2024-05-18 19:59:26 +02:00
Michael Eischer
aeb7eb245c retry: do not retry permanent errors
This is currently gated behind a feature flag as some unexpected
interactions might show up in the wild.
2024-05-18 19:59:26 +02:00
Michael Eischer
bf8cc59889 Use generic backend-error-redesign feature flag instead of http-timeouts
An individual flag for each change of the backend error handling would
be too finegrained. Thus, add a generic flag.
2024-05-18 19:54:52 +02:00
Michael Eischer
4740528a0b backend: add tests for IsPermanentError 2024-05-18 19:54:52 +02:00
Michael Eischer
6a85df7297 backend: add IsPermanentError() method to interface 2024-05-18 19:54:52 +02:00
Michael Eischer
cfc420664a mem: stricter handling of out of bounds requests 2024-05-18 19:54:52 +02:00
Michael Eischer
d40f23e716 azure/b2/gs/s3/swift: adapt cloud backend 2024-05-18 19:54:51 +02:00
Michael Eischer
e793c002ec local: stricter handling of short files 2024-05-18 19:54:21 +02:00
Michael Eischer
b4895ebd76 rest: rework error reporting and report too short files 2024-05-18 19:54:21 +02:00
Michael Eischer
eaa3f81d6b sftp: check for truncated files without an extra backend request 2024-05-18 19:54:21 +02:00
Michael Eischer
c6d74458ee sftp: improve handling of too short files 2024-05-18 19:54:21 +02:00
Michael Eischer
7ed560a201
Merge pull request #4796 from MichaelEischer/parallel-dump-load
dump: Parallelize loading large files
2024-05-14 22:35:44 +02:00
Michael Eischer
92221c2a6d
Merge pull request #4708 from zmanda/windows-securitydesc
Back up and restore SecurityDescriptors on Windows
2024-05-12 14:14:39 +00:00
Michael Eischer
b5fdb1d637
Merge pull request #4782 from MichaelEischer/fix-sftp-performance
Fix sftp upload performance
2024-05-12 15:28:33 +02:00
Michael Eischer
e4f9bce384
Merge pull request #4792 from restic/request-watchdog
backend: enforce that backend HTTP requests make progress
2024-05-09 23:55:30 +02:00
Michael Eischer
3740700ddc add http timeouts to changelog 2024-05-09 23:46:17 +02:00
Michael Eischer
ebd01a4675 backend: add tests for watchdogRoundTripper 2024-05-09 23:46:17 +02:00
Michael Eischer
8778670232 backend: cancel stuck http requests
requests that make no upload or download progress within a timeout are
canceled.
2024-05-09 23:46:17 +02:00
Michael Eischer
0987c731ec backend: configure protocol-level connection health checks
This should detect a connection that is stuck for more than 2 minutes.
2024-05-09 23:46:17 +02:00
aneesh-n
a4fd1b91e5
Fix review comments
Change lowerPrivileges from bool to atomic.Bool.
Add missing cleanup from upstream go-winio.
Add handling for ERROR_NOT_ALL_ASSIGNED warning.
2024-05-06 16:54:08 -06:00
Michael Eischer
e184538ddf dump: add changelog 2024-05-05 12:12:21 +02:00
Michael Eischer
4d55a62ada bloblru: add test for GetOrCompute 2024-05-05 12:00:25 +02:00
Michael Eischer
7cce667f92 fuse: switch to use bloblru.GetOrCompute 2024-05-05 11:38:42 +02:00
Michael Eischer
bd03af2feb dump: add GetOrCompute to bloblru cache 2024-05-05 11:38:42 +02:00
Michael Eischer
45509eafc8 dump: load blobs of a file from repository in parallel 2024-05-05 11:38:42 +02:00
Michael Eischer
24c1822220
Merge pull request #4794 from flow-c/master
Update 060_forget.rst
2024-05-04 08:25:06 +00:00