TrueCloudLab/restic
16
0
Fork 2
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions
8661 commits 9 branches 49 tags 79 MiB
Commit graph

2336 commits

Author SHA1 Message Date
Michael Eischer
36c4475ad9 rest: improve handling of HTTP2 goaway 
The HTTP client can only retry HTTP2 requests after receiving a GOAWAY
response if it can rewind the body. As we use a custom data type,
explicitly provide an implementation of `GetBody`.
 2024-08-26 15:44:17 +02:00
Michael Eischer
b8f409723d make timeout for slow requests configurable 2024-08-26 14:14:43 +02:00
Michael Eischer
311b27ced8 restic: cleanup redundant code in test case 2024-08-25 23:18:55 +02:00
Michael Eischer
43b36ad2b0 restore: test timestamps for macOS resource forks are restored correctly 2024-08-25 23:18:55 +02:00
Michael Eischer
2e55209b34 restic: restore timestamps after extended attributes 
restoring the xattr containing resource forks on macOS apparently
modifies the file modification timestamps. Thus, restore the timestamp
after xattrs.
 2024-08-25 23:18:55 +02:00
Michael Eischer
5cca6e66be
Merge pull request #4981 from konidev20/fix-gh-4934-cleanup-removed-snaphots-from-cache 
cache: clear snapshot files from cache during load index
 2024-08-16 19:04:59 +00:00
Srigovind Nayak
88174cd0a4
cache: remove redundant index file cleanup 
addressing code review comments
 2024-08-17 00:21:49 +05:30
Srigovind Nayak
b7d014b685
Revert "repository: removed redundant prepareCache method from Repository" 
This reverts commit 720609f8ba.
 2024-08-17 00:18:13 +05:30
Michael Eischer
74d3f92cc7
Merge pull request #4993 from MichaelEischer/fix-timeout-error 
backend: return correct error on upload/request timeout
 2024-08-15 22:07:37 +02:00
Michael Eischer
80f24584a5
Merge pull request #4998 from zmanda/ea_vss_fix 
Fix extended attributes handling for VSS snapshots
 2024-08-15 19:51:35 +00:00
Michael Eischer
8e00158c34
Merge pull request #5000 from deining/fix-typo 
Fix typos
 2024-08-15 19:42:14 +00:00
Michael Eischer
36b5580c1c
Merge pull request #4989 from plant99/progress-bar-for-restore-verify 
restore: Add progress bar to 'restore --verify'
 2024-08-15 19:34:05 +00:00
aneesh-n
19f487750e
Add test cases and handle volume GUID paths 
Gracefully handle errors while checking for EA and add debug logs.
 2024-08-11 19:25:58 -06:00
Shivashis Padhi
f1407afd1f
restore: Add progress bar to 'restore --verify' 2024-08-11 22:25:21 +02:00
Andreas Deininger
4401265e36 Fix typos 2024-08-11 21:38:15 +02:00
Srigovind Nayak
5fd984ba6f
cache: add test for the automated cache clear to cache backend 2024-08-11 23:41:07 +05:30
Srigovind Nayak
720609f8ba
repository: removed redundant prepareCache method from Repository 
* remove the prepareCache method from the Repository
* changed the signature of the SetIndex function to no longer return an error
 2024-08-11 23:41:07 +05:30
Srigovind Nayak
a23e7bfb82
cache: check for context cancellation before clearing cache 2024-08-11 23:41:07 +05:30
Srigovind Nayak
f66624f5bf
cache: backend add List method and a cache clear functionality 
* removes files which are no longer in the repository, including index files, snapshot files and pack files from the cache.

cache: fix ids set initialisation with NewIDSet()
 2024-08-11 23:40:52 +05:30
aneesh-n
849c441455
Gracefully handle invalid prepared volume names 2024-08-11 01:48:25 -06:00
aneesh-n
1d392a36f9
Fix extended attributes handling for VSS snapshots 2024-08-11 01:23:47 -06:00
Michael Eischer
3f5e2160de
Merge pull request #4938 from MichaelEischer/bump-go-version 
Bump go version to 1.21
 2024-08-10 19:57:59 +02:00
Michael Eischer
283225f15f
Merge pull request #4994 from MichaelEischer/less-flaky-watchdog-test 
backend: tweak timeouts to make watchdog timeout test less flaky
 2024-08-10 19:35:38 +02:00
Michael Eischer
400ae55940 replace deprecated usages of math/rand 2024-08-10 19:34:49 +02:00
Michael Eischer
0b19f6cf5a Switch back to sha256 from the std library 
The std library now also supports the sha assembly instructions on
ARM64. Thus, sha256-simd no longer provides a performance benefit.
 2024-08-10 19:16:10 +02:00
Michael Eischer
ad48751adb bump required go version to 1.21 2024-08-10 19:16:10 +02:00
Michael Eischer
86390b453d
Merge pull request #4980 from zmanda/unsupported-ea-handling 
Skip ExtendedAttribute processing in Windows for volumes that do not support EA
 2024-08-10 17:11:12 +00:00
Michael Eischer
fa35e72214 backend: tweak timeouts to make watchdog timeout test less flaky 2024-08-10 19:08:03 +02:00
Michael Eischer
05571286b2
Merge pull request #4946 from mikix/json-errors 
Improve error handling in --json mode
 2024-08-10 17:04:56 +00:00
aneesh-n
18e9d71d7a
Fix review comments 2024-08-10 10:38:04 -06:00
Michael Eischer
853a686994 backend: return correct error on upload/request timeout 2024-08-10 18:06:24 +02:00
Michael Eischer
a164789321
Merge pull request #4977 from MichaelEischer/warn-on-incomplete-metadata 
backup: warn but store store item if extended metadata is incomplete
 2024-08-10 17:59:25 +02:00
aneesh-n
71632a8197
Handle extended length paths 2024-08-05 16:03:43 -06:00
aneesh-n
85639f5159
Add handling for relative paths, vss paths, UNC paths 2024-08-04 13:19:13 -06:00
aneesh-n
c13725b5d0
Check EA support only for volumes, files and dirs 2024-08-04 11:05:40 -06:00
aneesh-n
89712f6640
Formatted 2024-08-04 10:36:13 -06:00
aneesh-n
9dedba6dfc
Address review comments 2024-08-04 10:23:39 -06:00
aneesh-n
8c8a066c0e
Correct the bitmask for fileSupportsExtendedAttributes and add link 2024-08-03 18:06:47 -06:00
aneesh-n
56113a8da7
Skip EA processing for volumes that do not support EA 2024-08-03 16:03:30 -06:00
Michael Eischer
73c9780321 backup: store but warn if extended metadata for item is incomplete 
Files were not included in the backup if the extended metadata for the
file could not be read. This is rather drastic. Instead settle on
returning a warning but still including the file in the backup.
 2024-08-03 22:25:29 +02:00
Michael Terry
88f59fc2d6 json: switch backup and restore errors from string to struct types 
This keeps backwards compatibility with the previous empty structs.
And maybe we'd want to put other fields into the inner struct later,
rather than the outer message.
 2024-08-03 16:21:44 -04:00
Michael Eischer
a48baf6f3a
Merge pull request #4976 from textaligncenter/backup-panic 
fix panic in fs_reader
 2024-08-03 19:56:15 +00:00
Michael Terry
a376323331 restore: print JSON versions of errors in --json mode 
Previously, they were printed as freeform text.

This also adds a ui.Terminal interface to make writing
tests easier and also adds a few tests.
 2024-08-03 15:18:46 -04:00
Michael Eischer
e622135e7e
Merge pull request #4974 from MichaelEischer/fix-cache-debug-log 
cache: fix confusing debug log
 2024-08-03 21:12:32 +02:00
textaligncenter
d8ea178e69 fix panic in fs_reader 2024-08-03 19:08:04 +00:00
Michael Terry
ad2585af67 backup: show actual error strings in --json mode 
Previously, an error JSON fragment would look like:
{"message_type": "error", "error": {}}

This is because encoding/json cannot marshal an error interface.
Instead, we now call .Error() to get the string value.
 2024-08-03 15:07:52 -04:00
Michael Terry
f4bdfea1c9 backup: print scanner errors to stderr, not stdout 2024-08-03 15:07:52 -04:00
Michael Eischer
d2f7c5a9c6
Merge pull request #4978 from konidev20/fix-gh-4949-repair-index-spurious-index 
rewrite: skip saving empty indexes during MasterIndex.Rewrite
 2024-08-03 18:53:57 +00:00
Srigovind Nayak
068d5b95c3
rewrite: skip saving empty indexes during MasterIndex.Rewrite 2024-08-03 23:34:59 +05:30
Michael Eischer
d4db5a364e
Merge pull request #4958 from mikix/restore-errors 
restore: clean up error handling when restoring metadata
 2024-08-03 17:58:54 +00:00
Michael Eischer
4266dca1b6 cache: fix confusing debug log 2024-08-03 18:51:38 +02:00
Michael Eischer
d407abb50f
Merge pull request #4960 from MichaelEischer/reduce-cancelation-delays 
Add more checks for canceled contexts
 2024-08-03 18:23:20 +02:00
Michael Eischer
ae1cb889dd Add more checks for canceled contexts 2024-07-31 19:30:47 +02:00
Michael Terry
6a97833337 restore: clean up error handling when restoring metadata 
- Fix a logic error that instead of reporting the *first*
  metadata-setting error that appears, we were instead reporting the
  *last* error (and only if the lchown call failed!).
- Don't show any errors when setting metadata for files in non-root
  mode (things like timestamps, attributes). Previously, only lchown
  errors were skipped. But other kinds of attribute errors make sense
  to skip as well. The code path happened to work correctly before
  because of the above logic error. But once that was fixed, this
  change needed to happen too.
 2024-07-30 19:27:34 -04:00
Michael Eischer
a9be986782 restorer: add minimal long path handling test 2024-07-29 21:11:47 +02:00
Michael Eischer
62c4a5e9a0 fs: fix long path handling for ResetPermissions, ClearAttributes and security descriptors 2024-07-29 20:10:58 +02:00
Michael Eischer
729e009485 replace all uses of target in relation to backup 2024-07-24 19:40:53 +02:00
Michael Eischer
fa5ff0873a
Merge pull request #4925 from MichaelEischer/fix-restore-truncation 
restore: Fix truncation of uptodate files
 2024-07-21 12:28:12 +02:00
Michael Eischer
0dcac90bea restorer: don't track already uptodate blobs 2024-07-21 12:03:56 +02:00
Michael Eischer
10efa77103 restorer: add test for file truncation case 2024-07-21 12:03:56 +02:00
Michael Eischer
6f8e17a463 restorer: minor code cleanups 2024-07-21 12:03:56 +02:00
Michael Eischer
5d6ce59a8d restorer: also truncate files if their content is already uptodate 
Files for which no blobs have to be restored, still have to be truncated
to the correct size. Take a file with content "foobar" that should be
replaced by restore with content "foo". The first three bytes are
already uptodate, such that no data has to be written. As file
truncation normally happens when writing data, a special case is
necessary.

This no blobs written special case is unified with the empty file
special case.
 2024-07-21 12:03:56 +02:00
Michael Eischer
59c880fce3 restorer: only show size in text output for files 2024-07-20 12:46:28 +02:00
Alex Johnson
3bf2927006 Update snapshot summary on rewrite 
Signed-off-by: Alex Johnson <hello@alex-johnson.net>
 2024-07-16 12:06:50 -05:00
Michael Eischer
dcfffd7779 restore: extend overwrite test for small files 2024-07-14 11:30:41 +02:00
Michael Eischer
44e3610b32 restore: progress bar total on windows 2024-07-14 11:21:11 +02:00
Michael Eischer
98cfb2c4c8 restore: test progress reporting for partially up to date files 2024-07-14 11:21:11 +02:00
Michael Eischer
26aa65e0d4 restore: add regression test for corrupt in-place restore of large file 2024-07-14 11:21:11 +02:00
Michael Eischer
2833b2f699 restore: fix progress bar for partially up to date files 2024-07-14 11:21:11 +02:00
Michael Eischer
c77b2d5ca2 restore: avoid long cancelation delay for frequently used blobs 2024-07-14 11:21:11 +02:00
Michael Eischer
f64191da9c restore: improve reporting of cancelation errors 2024-07-14 11:21:08 +02:00
Michael Eischer
2971a769da restore: fix corrupt restore of partially up to date files 2024-07-12 22:27:00 +02:00
Michael Eischer
40e5163114 restore: properly cancel file verification 2024-07-12 22:26:22 +02:00
Michael Eischer
31887ea9aa restore: fix hang on command cancelation 2024-07-12 22:24:35 +02:00
Michael Eischer
375c572c4f
Merge pull request #4884 from MichaelEischer/lock-and-not-exist-error-codes 
Return different exit code if repo is locked or does not exist
 2024-07-12 21:05:52 +02:00
Michael Eischer
8731667156 restore: do not delete target if it is a file 
This is implicitly achieved by creating the target directory as the
first step, which will fail if a file already exist in its place.
 2024-07-10 22:08:49 +02:00
Michael Eischer
94fdca08c4 return exit code 10 if repository does not exist 2024-07-10 21:46:26 +02:00
Michael Eischer
d8dbc71deb
Merge pull request #4908 from MichaelEischer/improve-anonymous-s3 
S3: Improve anonymous access
 2024-07-10 20:19:23 +02:00
Michael Eischer
f74e70cc36 s3: forbid anonymous authentication unless explicitly requested 2024-07-10 20:10:27 +02:00
Michael Eischer
4b364940aa s3: use http client with configured timeouts for s3 IAM communication 
The default client has no timeouts configured opening network
connections. Thus, if 169.254.169.254 is inaccessible, then the client
would wait for until the operating system gives up, which will take
several minutes.
 2024-07-07 11:32:40 +02:00
Michael Eischer
a2a2401a68 s3: prevent repeated credential queries with anonymous authentication 2024-07-07 11:31:04 +02:00
Michael Eischer
aedead2823 ui/termstatus: fix clearing status lines 
To clear the status lines, they should be set to an empty array to
prevent future updates of those lines. Setting the status lines to an
array containing an empty string is wrong as this causes the output to
continuously add that empty status line after each message.
 2024-07-06 11:27:35 +02:00
Michael Eischer
868219aad1 restore: test --dry-run plus --delete 2024-07-05 22:38:39 +02:00
Michael Eischer
aa8e18cf32 restore: add deletions to progress output 2024-07-05 22:38:39 +02:00
Michael Eischer
f4b15fdd96 restore: allow deleting a directory to replace it with a file 
When the `--delete` option is specified, recursively delete directories
that should be replaced with a file.
 2024-07-05 22:38:39 +02:00
Michael Eischer
168fc09d5f restore: use case insensitive file name comparison on windows 2024-07-05 22:38:39 +02:00
Michael Eischer
013a6156bd restore: remove unused parameter from SelectFilter 2024-07-05 22:38:39 +02:00
Michael Eischer
ac44bdf6dd restore: add --delete option to remove files that are not in snapshot 2024-07-05 22:38:39 +02:00
Michael Eischer
144e2a451f restore: track expected filenames in a folder 2024-07-05 21:03:35 +02:00
Michael Eischer
d762f4ee64 restore: simplfy selectFilter arguments 2024-07-05 21:03:35 +02:00
Michael Eischer
a9a60f77ce restore: optimize memory usage 2024-07-05 21:03:35 +02:00
Michael Eischer
1369658a32 archiver: extract Readdirnames to fs package 2024-07-05 21:03:34 +02:00
Michael Eischer
ae978d60cc restore: pass action enum to restore progress 2024-07-05 20:41:27 +02:00
Michael Eischer
83351f42e3 restore: add dry-run support 2024-07-05 20:41:27 +02:00
Michael Eischer
c47bf33884 restore: print processed files at maximum verbosity 2024-07-05 20:41:27 +02:00
Michael Eischer
c9a4a95848 check: suggest using repair packs to repair truncated pack files 
Previously, that help message was only shown for running `check
--read-data`.
 2024-07-05 20:04:25 +02:00
Viktor Szépe
ac00229386 Fix typos 2024-07-03 20:02:06 +02:00
Michael Eischer
132965fd5c
Merge pull request #4864 from MichaelEischer/restore-delete 
restore: improve file type mismatch handling
 2024-06-26 21:27:57 +02:00
Michael Eischer
9817126341
Merge pull request #4863 from MichaelEischer/safer-repo-init2 
repository: prevent initialization if a snapshot exists
 2024-06-26 21:11:11 +02:00
Michael Eischer
c01b655229 backup: fix reading xattrs on Windows for root disks as regular user 
Unprivileged users cannot open the root disks with write permissions for
xattrs. Thus, only request read permissions for reading the xattrs.
 2024-06-26 20:13:50 +02:00
Michael Eischer
20a6c526e5 restic: also fix file path when reading windows xattrs 2024-06-14 22:40:25 +02:00
Michael Eischer
b80aa7b1cc repository: prevent initialization if a snapshot exists 2024-06-14 20:37:01 +02:00
Michael Eischer
deca7d08ac restorer: cleanup unexpected xattrs on windows 2024-06-14 20:17:06 +02:00
Michael Eischer
9572b7224f restorer: windows test fixes 2024-06-13 23:20:09 +02:00
Michael Eischer
ca41c8fd11 restore: use fs function wrappers 
This ensures proper path handling on Windows.
 2024-06-13 22:40:37 +02:00
Michael Eischer
4d6042fe95 restore: remove unexpected xattrs from files 2024-06-13 22:21:00 +02:00
Michael Eischer
ebbd4e26d7 restorer: allow directory to replace existing file 2024-06-13 21:57:48 +02:00
Michael Eischer
ac729db3ce restorer: fix overwriting of special file types 
An attempt to replace an existing file with a hardlink previously ended
with a missing file.

Remove an existing file before trying to restore a special node. This
generalizes the existing behavior for symlinks to all special node
types.
 2024-06-13 21:56:18 +02:00
Michael Eischer
c7902b7724 restorer: cleanup overwrite tests 2024-06-13 21:55:33 +02:00
Michael Eischer
c598a751c2 restore: fine-grained sparse support for windows 2024-06-13 21:55:33 +02:00
Michael Eischer
d265ec64f2 restore: correctly handle existing hardlinks 
With hardlinks there's no efficient way to detect which files are linked
with each other. Thus, just start from scratch when restore has to
modify a hardlinked file.
 2024-06-13 21:55:33 +02:00
Michael Eischer
f19b69af25 restore: allow overwrite to replace empty directories and symlinks 
With an already existing file tree an old directory or symlink may exist
in a place where restore wants to create a new file. Thus, check for
unexpected file types and clean up if necessary.
 2024-06-13 21:55:33 +02:00
Michael Eischer
f1c76a8286 restore: fix corrupted sparse files 2024-06-13 21:17:30 +02:00
Michael Eischer
1eccd6504b restore: test restore only changed parts functionality 2024-06-13 21:17:30 +02:00
Michael Eischer
5c3709e17a restore: add --overwrite=if-changed to skip files if their mtime&size matches 
--overwrite=always still checks the file content
 2024-06-13 21:17:30 +02:00
Michael Eischer
a66658b4c9 restore: only restore changed file parts 
For files that should be overwritten, first check whether their content
matches the snapshot and only restore diverging parts.
 2024-06-13 21:17:30 +02:00
Michael Eischer
62e0e5e8d8 restore: cleanup options handling 2024-06-12 22:53:53 +02:00
Michael Eischer
8733cd21b9 restore: fix file encryption on windows 
The assumption that a file is not readonly is no longer valid.
 2024-06-12 22:53:53 +02:00
Michael Eischer
8aa3ab6ef8 restore: check file type in verify 2024-06-12 22:53:53 +02:00
Michael Eischer
7f7c995977 fix linter warnings 2024-06-12 22:36:52 +02:00
Michael Eischer
ba53a2abb5 test overwrite behavior 2024-06-12 22:36:52 +02:00
Michael Eischer
e47e08a688 restorer: separately track skipped files 2024-06-12 22:36:52 +02:00
Michael Eischer
64b7b6b975 restore/ui: refactor for extensibility 2024-06-12 22:36:52 +02:00
Michael Eischer
6a4ae9d6b1 restore: configurable overwrite behavior 2024-06-12 22:36:52 +02:00
Michael Eischer
a23cb3a428 restore: reduce memory usage 2024-06-12 22:36:52 +02:00
Michael Eischer
2b50c2606c restorer: use options struct 2024-06-12 22:36:52 +02:00
Michael Eischer
0fcd89f892 restorer: remove special case for blobs with many occurrences 
Loading blobs by now is no longer prone to timeouts when processing
takes a long time.
 2024-06-12 22:36:52 +02:00
Michael Eischer
fd2ff464a2 restorer: remove stale comment 2024-06-12 22:36:52 +02:00
Michael Eischer
607daeed4f restore: move nil pointer check into restoreui 2024-06-12 22:36:52 +02:00
Michael Eischer
30320a249a restore: let filerestorer also handle empty files 
This get's rid of the corresponding special cases.
 2024-06-12 22:36:52 +02:00
Michael Eischer
c166ad7daf restore: factor out file creation helper 2024-06-12 22:36:52 +02:00
Michael Eischer
da4512738a
Merge pull request #4853 from MichaelEischer/safer-repo-init 
init: double check that no repository exists yet
 2024-06-09 11:27:45 +02:00
Michael Eischer
8440b94159
Merge pull request #4807 from zmanda/windows-extendedattribs 
Back up and restore Extended Attributes on Windows NTFS
 2024-06-09 09:18:04 +00:00
Michael Eischer
bab760369f repository: double check that there is not repository before init 
Apparently, calling `Stat` on the config file can be unreliable for some
backends.
 2024-06-09 00:05:32 +02:00
Michael Eischer
78485160fc
Merge pull request #4849 from fthoma/table-tcwidth 
Use character display width for table padding
 2024-06-07 19:53:14 +00:00
Michael Eischer
7cd324fe26 ui/table: avoid duplicate table cell width calculation 2024-06-07 21:45:40 +02:00
Michael Eischer
edd3e214c2 ui/table: fix width calculation of multi-line column headers 2024-06-07 21:44:49 +02:00
Michael Eischer
3f878aa8e7
Merge pull request #4845 from greatroar/errors 
Fix error handling bug + clean up error messages
 2024-06-07 17:07:07 +00:00
Michael Eischer
112de8526d
Merge pull request #4852 from MichaelEischer/master 
crypto: cleanup ciphertext verification error
 2024-06-07 19:06:08 +02:00
Michael Eischer
0a70bbcea5
Merge pull request #4844 from MichaelEischer/improve-timeout-error 
backend: Improve timeout error message
 2024-06-07 19:05:39 +02:00
aneesh-n
4f053da06a
Fix test case by correcting cleanup order 2024-06-05 23:10:29 -06:00
aneesh-n
2101dfe448
Add missing return to fix failing test 2024-06-05 22:40:21 -06:00
aneesh-n
7a48c9ebd7
Fix review comments 2024-06-05 16:06:57 -06:00
Michael Eischer
b82ecc6425 crypto: cleanup ciphertext verification error 2024-06-05 22:33:20 +02:00
Florian Thoma
e9de9684f4
Use character display width for table padding 
Using len(...) for table cell padding produced wrong results for unicode
chracters leading to misaligned tables. Implementation changed to take
the actual terminal display width into consideration.
 2024-06-05 09:33:15 +02:00
Leo R. Lundgren
b2bbbe805f azure: Improve error message in azure.Create() 2024-06-03 23:37:17 +02:00
Michael Eischer
660679c2f6
Merge pull request #4835 from MichaelEischer/fix-list-cancel 
backend/retry: do not log final error if context was canceled
 2024-06-01 19:17:30 +02:00
Michael Eischer
db2398f35b backend: increase request progress timeout to 5 minutes 
Apparently, 2 minutes are too short in some cases and can result in
canceled List requests.
 2024-06-01 19:01:51 +02:00
Michael Eischer
6bf3d4859f backend: improve error on http request timeout 
Now yields a "request timeout" error instead of "context canceled".
 2024-06-01 18:52:39 +02:00
greatroar
4a874000b7 gs: Replace some errors.Wrap calls 
The first one in Create is already a WithStack error. The rest were
referencing code that hasn't existed for quite some time. Note that
errors from Google SDKs tends to start with "google:" or "googleapi:".

Also, use restic/internal/errors.
 2024-06-01 15:11:06 +02:00