TrueCloudLab/restic
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions
8553 commits 9 branches 49 tags 79 MiB
Commit graph

8553 commits

This branch
This branch
All branches
Author SHA1 Message Date
Michael Eischer
20d8eed400 repository: streamPack: separate requests for gap larger than 1MB 
With most cloud providers, traffic is much more expensive than API
calls. Thus slightly bias streamPack towards a bit more API calls in
exchange for slightly less traffic.
 2024-04-22 21:21:23 +02:00
Michael Eischer
cf700d8794 repository: streamPack: reuse zstd decoder 2024-04-22 21:21:23 +02:00
Michael Eischer
666a0b0bdb repository: streamPack: replace streaming with chunked download 
Due to the interface of streamPack, we cannot guarantee that operations
progress fast enough that the underlying connections remains open. This
introduces partial failures which massively complicate the error
handling.

Switch to a simpler approach that retrieves the pack in chunks of 32MB.
If a blob is larger than this limit, then it is downloaded separately.

To avoid multiple copies in memory, an auxiliary interface
`discardReader` is introduced that allows directly accessing the
downloaded byte slices, while still supporting the streaming used by the
`check` command.
 2024-04-22 21:21:23 +02:00
Michael Eischer
621012dac0 repository: Add blob loading fallback to LoadBlobsFromPack 
Try to retrieve individual blobs via LoadBlob if streaming did not work.
 2024-04-21 21:35:55 +02:00
Michael Eischer
6c6dceade3 global: unify backend open and create 2024-04-19 22:26:14 +02:00
Michael Eischer
10355c3fb6 repository: Better error message if blob is larger than 4GB 2024-04-19 22:00:35 +02:00
Michael Eischer
228b35f074
Merge pull request #4769 from will-ca/patch-1 
Tiny wording clarification in `restic-stats.1`.
 2024-04-18 19:00:39 +00:00
will-ca
6aced61c72
Tiny docs wording clarification. 2024-04-18 07:29:55 +00:00
Michael Eischer
4d22412e0c
Merge pull request #4766 from coderwander/master 
Fix struct names
 2024-04-18 06:18:19 +00:00
coderwander
a82ed71de7 Fix struct names 
Signed-off-by: coderwander <770732124@qq.com>
 2024-04-18 10:02:09 +08:00
Michael Eischer
2173c69280
Merge pull request #4770 from testwill/close_files 
fix: close files
 2024-04-17 16:50:20 +00:00
Michael Eischer
001bb71676 repair packs: Properly close backup files 2024-04-17 18:32:30 +02:00
Michael Eischer
c9191ea72c forget: cleanup verbose output on snapshot deletion error 2024-04-14 14:17:40 +02:00
Michael Eischer
09587e6c08 repository: duplicate a few blobs in prune tests 2024-04-14 13:57:19 +02:00
Michael Eischer
defd7ae729 prune/repair index: reset in-memory index after command 
The current in-memory index becomes stale after prune or repair index
have run. Thus, just drop the in-memory index altogether once these
commands have finished.
 2024-04-14 13:46:24 +02:00
Michael Eischer
038586dc9d repository: add minimal test for prune 2024-04-14 13:45:17 +02:00
Michael Eischer
d8622c86eb prune: clean up internal interface 2024-04-14 13:45:15 +02:00
Michael Eischer
8d507c1372 repository: add basic test for RepairIndex 2024-04-14 13:45:15 +02:00
Michael Eischer
310db03c0e repair index: improve log output if index cannot be deleted 
The operation will always fail with an error if an index cannot be
deleted. Thus, this change is purely cosmetic.
 2024-04-14 13:45:13 +02:00
Michael Eischer
7d1b9cde34 repository: use normal Init method in tests 2024-04-14 13:45:11 +02:00
Michael Eischer
b25fc2c89d repository: remove redundant flushes from tests 2024-04-14 13:45:10 +02:00
Michael Eischer
c65459cd8a repository: speed up tests 2024-04-14 13:45:10 +02:00
Michael Eischer
eda9f7beb4 ui/progress: add helper to print messages during tests 2024-04-14 13:45:08 +02:00
Michael Eischer
35277b7797 backend/mem: cleanup not found error message 2024-04-14 13:45:06 +02:00
Michael Eischer
7ba5e95a82 check: allow tests to only verify pack&index integrity 2024-04-14 13:45:04 +02:00
Michael Eischer
4c9a10ca37 repair packs: deduplicate index rebuild 2024-04-14 13:45:02 +02:00
Michael Eischer
85e4021619 prune: move additional option checks to repository 2024-04-14 13:44:58 +02:00
Michael Eischer
55d56db31b
Merge pull request #4743 from MichaelEischer/deprecate-s3legacy-layout 
Deprecate s3legacy layout
 2024-04-11 22:09:34 +02:00
Michael Eischer
fc3b548625 prune: move logic into repository package 2024-04-10 21:30:52 +02:00
Michael Eischer
df9d4b455d prune: prepare for moving code to repository package 2024-04-10 21:30:52 +02:00
Michael Eischer
866ddf5698 repair index: refactor code into repository package 2024-04-10 21:30:52 +02:00
Michael Eischer
32a234b67e prune/forget/repair index: convert output to use progress.Printer 2024-04-10 21:30:52 +02:00
Michael Eischer
739d11c2eb forget: replace usage of DeleteFilesChecked 
This simplifies refactoring prune into the repository package.
 2024-04-10 21:30:52 +02:00
Michael Eischer
591b421c4a Deprecate s3legacy layout 2024-04-10 21:27:56 +02:00
Michael Eischer
8efc3a8b7d
Merge pull request #4668 from MichaelEischer/backup-xattr-parent-enoperm 
backup: Ignore xattr.list permission error for parent directories
 2024-04-10 21:25:28 +02:00
Michael Eischer
bf054c09d2 backup: Ignore xattr.list permission error for parent directories 
On FreeBSD, limited users may not be able to even list xattrs for the
parent directories above the snapshot source paths. As this can cause
the backup to fail, just ignore those errors.
 2024-04-10 20:46:15 +02:00
Michael Eischer
0747cf5319 cache: ignore ErrNotExist during cleanup of old files 
Two restic processes running concurrently can try to remove the same
files from the cache. This could cause one process to fail with an error
if the other one has already remove a file that the current process also
tries to delete.
 2024-04-10 19:25:51 +02:00
Michael Eischer
6091029fd6
Merge pull request #4756 from mgeisler/patch-1 
doc: fix typo in 047_tuning_backup_parameters.rst
 2024-04-07 19:24:09 +00:00
Martin Geisler
09d2183351
doc: fix typo in 047_tuning_backup_parameters.rst 2024-04-07 18:05:53 +02:00
Michael Eischer
a4b7ebecfc
Merge pull request #4750 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.40.0 
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
 2024-04-03 20:10:41 +00:00
dependabot[bot]
ba136b31b8
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0 
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.39.0 to 1.40.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.39.0...spanner/v1.40.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-04-03 20:03:22 +00:00
Michael Eischer
f328111a6e
Merge pull request #4751 from restic/dependabot/go_modules/golang.org/x/net-0.23.0 
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
 2024-04-03 19:52:29 +00:00
Michael Eischer
9fb017e67a
Merge pull request #4745 from MichaelEischer/full-id-key-list 
key list: include full key id in JSON output
 2024-04-03 21:50:02 +02:00
Michael Eischer
49f98f25fc
Merge pull request #4742 from MichaelEischer/consistent-rtest-import 
Use consistent alias for interal/test package
 2024-04-03 21:47:32 +02:00
dependabot[bot]
96c602a6de
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0 
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-04-03 19:41:38 +00:00
Michael Eischer
1d0a20dd58
Merge pull request #4748 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.10.0 
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.9.2 to 1.10.0
 2024-04-03 19:34:47 +00:00
Michael Eischer
6cca1d5705
Merge pull request #4655 from ae-govau/unixsocket 
Enhancement: option to send HTTP over unix socket
 2024-04-03 19:29:21 +00:00
dependabot[bot]
f8a72ac2a3
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore 
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go) from 1.9.2 to 1.10.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.2...sdk/azcore/v1.10.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-04-01 02:32:19 +00:00
Michael Eischer
5145c8f9c0 key list: include full key id in JSON output 2024-03-31 12:25:20 +02:00
Michael Eischer
831fc4413d
Merge pull request #4737 from stephan0307/3117 
json output forget command: added id's in snapshots within reasons object
 2024-03-29 13:11:24 +00:00