TrueCloudLab/restic
16
0
Fork 3
Code Issues 2 Pull requests 2 Projects Releases Packages Wiki Activity Actions
6837 commits 9 branches 49 tags 79 MiB
Commit graph

6837 commits

This branch
This branch
All branches
Author SHA1 Message Date
Michael Eischer
0b7291b8b2 mount: Fix parent inode used by snapshots dir 2022-08-07 13:03:32 +02:00
greatroar
cfa80e2c6b mount: remove unused inode field from root node 2022-08-07 13:03:26 +02:00
MichaelEischer
74ae76036f
Merge pull request #2913 from aawsome/mount-snapshot-slashes 
mount: Make snapshots dir structure customizable
 2022-08-07 12:27:59 +02:00
MichaelEischer
09497aec02
Merge pull request #3826 from MichaelEischer/debug-log-for-release 
Debug log for release build
 2022-08-07 12:15:27 +02:00
Michael Eischer
83b4c50ee3 Mention --snapshot-template and --time-template in changelog 2022-08-07 12:13:06 +02:00
Michael Eischer
caa17988a3 fuse: Redesign snapshot dirstruct 
Cleanly separate the directory presentation and the snapshot directory
structure. SnapshotsDir now translates the dirStruct into a format
usable by the fuse library and contains only minimal special case rules.
All decisions have moved into SnapshotsDirStructure which now creates a
fully preassembled tree data structure.
 2022-08-07 12:13:06 +02:00
Michael Eischer
a3e48da3a3 Add changelog for DEBUG_LOG available in release builds 2022-08-05 23:49:39 +02:00
Michael Eischer
b3cdee66a9 update documentation to reflect DEBUG_LOG for release builds 2022-08-05 23:49:39 +02:00
Michael Eischer
1ed775e3a8 debug: support roundtripper logging also for release builds 
Different from debug builds do not use the eofDetectRoundTripper if
logging is disabled.
 2022-08-05 23:49:39 +02:00
Michael Eischer
38becfc436 debug: enable debug support for release builds 2022-08-05 23:49:39 +02:00
Michael Eischer
82c268c917 Remove unused hooks mechanism 2022-08-05 23:49:39 +02:00
Michael Eischer
7266f07c87 repository: StreamPack in parts if there are too large gaps 
For large pack sizes we might be only interested in the first and last
blob of a pack file. Thus stream a pack file in multiple parts if the
gaps between requested blobs grow too large.
 2022-08-05 23:48:36 +02:00
Michael Eischer
55a11c1396 Reword prune --repack-small description 2022-08-05 23:48:36 +02:00
Michael Eischer
eaf43607f9 Add note that pack-size is not an exact limit 2022-08-05 23:48:36 +02:00
Michael Eischer
7f3b2be1e8 s3: Disable multipart uploads below 200MB 2022-08-05 23:48:36 +02:00
Michael Eischer
176b387d98 Always repack very small pack files 2022-08-05 23:48:36 +02:00
Michael Eischer
324935cb80 Only repack small files if there are multiple of them 2022-08-05 23:48:34 +02:00
Michael Eischer
1b076cda97 rename option to --pack-size 2022-08-05 23:47:43 +02:00
Michael Eischer
d7e2892048 Add changelog for packsize option 2022-08-05 23:47:43 +02:00
Michael Eischer
8a44258b6f update restic help snippets in documentation 2022-08-05 23:47:43 +02:00
Michael Eischer
420ddc03c9 rework pack size parameter documentation 2022-08-05 23:47:43 +02:00
Kyle Brennan
e43be84eb8 document minPackSize 2022-08-05 23:47:41 +02:00
Kyle Brennan
1e3f05c3f1 repository: prevent header overfill 2022-08-05 23:47:12 +02:00
Michael Eischer
6a6d313c9a prune: reduce priority of repacking small packs 2022-08-05 23:47:12 +02:00
Kyle Brennan
0269381b8d prune: add repack-small parameter 2022-08-05 23:47:12 +02:00
Michael Eischer
0a6fa602c8 add option for setting min pack size 2022-08-05 23:47:12 +02:00
Michael Eischer
2db7733ee3 fuse: remove unused MetaDir 2022-08-05 23:46:46 +02:00
Michael Eischer
f678f7cb04 fuse: cleanup test 2022-08-05 23:46:46 +02:00
Alexander Weiss
1751afae26 Make snapshots dirs in mount command customizable 2022-08-05 23:46:46 +02:00
Alexander Weiss
57f4003f2f Generalize fuse snapshot dirs implemetation 
+ allow "/" in tags and snapshot template
 2022-08-05 23:46:46 +02:00
Alexander Weiss
696c18e031 Add possibility to set snapshot ID (used in test) 2022-08-05 23:46:46 +02:00
MichaelEischer
04a8ee80fb
Merge pull request #3829 from MichaelEischer/prune-refactor 
Split prune into slightly small functions
 2022-08-05 23:29:52 +02:00
MichaelEischer
15679be858
Merge pull request #3841 from MichaelEischer/compression-env 
Add environment variable RESTIC_COMPRESSION
 2022-08-05 23:20:04 +02:00
MichaelEischer
7d14b1baf1
Merge pull request #3845 from greatroar/solaris-xattr 
internal/restic: Handle EINVAL for xattr on Solaris
 2022-08-04 22:04:52 +02:00
greatroar
ad6ac680af internal/restic: Handle EINVAL for xattr on Solaris 
Also make the errors a bit less verbose by not prepending the operation,
since pkg/xattr already does that. Old errors looked like

    Listxattr: xattr.list /myfiles/.zfs/snapshot: invalid argument
 2022-08-01 12:45:17 +02:00
MichaelEischer
846d021db5
Merge pull request #3840 from greatroar/sftp-init 
Speed up restic init over slow SFTP links
 2022-07-31 19:54:59 +02:00
greatroar
b9fa6e05bd Add changelog for #3837/#3840 2022-07-30 22:53:52 +02:00
Michael Eischer
73053674d9 repository: Test fallback to existing blobs 2022-07-30 17:37:07 +02:00
Michael Eischer
e85a21eda2 prune: move code 2022-07-30 17:37:07 +02:00
Michael Eischer
623770eebb repository: try to recover from invalid blob while repacking 
If a blob that should be kept is invalid, Repack will now try to request
the blob using LoadBlob. Only return an error if that fails.
 2022-07-30 17:37:07 +02:00
Michael Eischer
d0590b7841 prune: Add internal integrity check 
After repacking every blob that should be kept must have been repacked.
We have seen a few cases in which a single blob went missing, which
could have been caused by a bitflip somewhere. This sanity check might
help catch some of these cases.
 2022-07-30 17:37:07 +02:00
Michael Eischer
5cbde03eae prune: split into smaller functions 2022-07-30 17:37:07 +02:00
Alexander Weiss
7643237da5 prune: separate collecting/printing/pruning 2022-07-30 17:37:07 +02:00
Michael Eischer
5723c72eb1 Add environment variable RESTIC_COMPRESSION 2022-07-30 16:21:53 +02:00
greatroar
2bdc40e612 Speed up restic init over slow SFTP links 
pkg/sftp.Client.MkdirAll(d) does a Stat to determine if d exists and is
a directory, then a recursive call to create the parent, so the calls
for data/?? each take three round trips. Doing a Mkdir first should
eliminate two round trips for 255/256 data directories as well as all
but one of the top-level directories.

Also, we can do all of the calls concurrently. This may reintroduce some
of the Stat calls when multiple goroutines try to create the same
parent, but at the default number of connections, that should not be
much of a problem.
 2022-07-30 13:09:08 +02:00
greatroar
23ebec717c Remove stale comments from backend/sftp 
The preExec and postExec functions were removed in
0bdb131521 from 2018.
 2022-07-30 13:07:25 +02:00
MichaelEischer
4ffd479ba4
Merge pull request #3773 from MichaelEischer/efficient-dir-json 
Reduce memory usage for large directories/files
 2022-07-23 17:47:32 +02:00
Michael Eischer
2ba14160de Add changelog for the optimized tree serialization 2022-07-23 14:49:08 +02:00
Michael Eischer
4a10ebed15 archiver: reduce memory usage for large files 
FutureBlob now uses a Take() method as a more memory-efficient way to
retrieve the futures result. In addition, futures are now collected
while saving the file. As only a limited number of blobs can be queued
for uploading, for a large file nearly all FutureBlobs already have
their result ready, such that the FutureBlob object just consumes
memory.
 2022-07-23 14:45:07 +02:00
Michael Eischer
b817681a11 archiver: Incrementally serialize tree nodes 
That way it is not necessary to keep both the Nodes forming a Tree and
the serialized JSON version in memory.
 2022-07-23 14:45:07 +02:00