TrueCloudLab/restic
16
0
Fork 2
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions
5921 commits 9 branches 49 tags 79 MiB
Commit graph

5921 commits

This branch
This branch
All branches
Author SHA1 Message Date
MichaelEischer
f1cfb97237
Merge pull request #3514 from phcreery/rclone_timeout 
rclone: extend timeout from 60s to 240s
 2021-11-07 18:10:34 +01:00
MichaelEischer
cb81ee9396
Merge pull request #3474 from kitone/fix-issue-3382 
Honor RESTIC_CACHE_DIR environment variable
 2021-11-07 17:57:54 +01:00
Michael Eischer
b0e64deb27 rclone: Fix timeout calculation 2021-11-07 17:49:33 +01:00
phcreery
43d173b042 rclone: add timeout option and documentation 2021-11-07 17:49:21 +01:00
rawtaz
1b152a2c4d
Merge pull request #3568 from MichaelEischer/fix-local-fd-leak 
local: Fix fd leak when encountering files directly inside data/
 2021-11-07 01:36:48 +01:00
MichaelEischer
15cc3c0e23
Merge pull request #3566 from mathstuf/check-progress-output-stomping 
check: wait for progress bar output
 2021-11-06 23:34:49 +01:00
kitone
5904f80cfa restic cache should display the name of the cache without shortening it in the case of the restic check 2021-11-06 20:18:51 +01:00
Ben Boeckel
4d579c4387 check: wait for progress bar output 
Further code will also output to the terminal and the bar's cursor
positioning causes its output to overlap with the remaining output in a
racy way.

Fixes: #3344
 2021-11-06 15:05:09 -04:00
Michael Eischer
15d7313387 local: Fix fd leak when encountering files directly inside data/ 2021-11-06 19:44:57 +01:00
MichaelEischer
6c84ea1412
Merge pull request #3548 from gum3ng/issue_3490 
Support for specifying amount of data in read-data-subset
 2021-11-05 23:28:11 +01:00
MichaelEischer
78c7dd53ef
Merge pull request #3526 from greatroar/dump-refactor 
Refactor internal/dump + concurrent load/write
 2021-11-05 22:38:39 +01:00
MichaelEischer
a34bfa8269
Merge pull request #3562 from restic/rawtaz-mount-usage 
mount: Improve usage information when mounted
 2021-11-05 22:10:32 +01:00
kitone
0425a30420 add changelog issue-3382 2021-11-04 15:13:48 +01:00
kitone
1b23675f21 cache --cleanup should handle directories created by restic check. 
Because there is no guarantee that a cleanup of these directories will occur
after the "restic check", we extend the behavior to detect and manage these
specific cache directories and allow their cleanup too.
 2021-11-04 15:10:38 +01:00
Gautam Menghani
836fbb9133 [#issue 3490] Support for specifying file size in read-data-subset 2021-11-02 15:25:46 +05:30
greatroar
c71729dfc4 Refactor internal/dump + concurrent load/write 
Package internal/dump has been reworked so its API consists of a single
type Dumper that handles tar and zip formats. Tree loading and node
writing happen concurrently.
 2021-11-01 23:01:55 +01:00
Leo R. Lundgren
711ceb0109 mount: Improve usage information when mounted 2021-11-01 20:59:20 +01:00
MichaelEischer
829c0a67af
Merge pull request #3520 from nxt-engineering/Docker 
Docker Multistage build
 2021-10-24 17:37:54 +02:00
Alexander Neumann
fb5d9345a7
Merge pull request #3510 from MichaelEischer/fix-archiver-early-on-abort 
archiver: Fix TestArchiverAbortEarlyOnError test
 2021-10-16 15:37:41 +02:00
kitone
95eb859b54 Honor RESTIC_CACHE_DIR environment variable 
Fix #3382: restic check doesn't obey the RESTIC_CACHE_DIR environment variable
 2021-10-10 16:00:02 +02:00
Michael Eischer
257740b0cc b2: Successful delete if file does not exist 
When deleting a file, B2 sometimes returns a "500 Service Unavailable"
error but nevertheless correctly deletes the file. Due to retries in
the B2 library blazer, we sometimes also see a "400 File not present"
error. The retries of restic for the delete request then fail with
"404 File with such name does not exist.".

As we have to rely on request retries in a distributed system to handle
temporary errors, also consider a delete request to be successful if the
file is reported as not existing. This should be safe as B2 claims to
provide a strongly consistent bucket listing and thus a missing file
shouldn't mysteriously show up again later on.
 2021-10-09 23:51:12 +02:00
MichaelEischer
46d08d9404
Merge pull request #3535 from jtagcat/writingverbose 
Change: selfupdate: 'writing restic to' as verbose
 2021-10-09 22:41:51 +02:00
jtagcat
a7853057ab Change: selfupdate: 'writing restic to' as verbose 
Running restic self-update --quiet no longer
prints "writing restic to /usr/local/bin/restic".

The only output printed with -q is failures or
"successfully updated restic to version 0.12.1"

https://github.com/restic/restic/pull/3535

fix test fail: changelog title can't end with `.`

shorten changelog title
 2021-10-09 23:24:18 +03:00
MichaelEischer
eb282532dc
Merge pull request #3534 from jtagcat/patch-1 
docs/example: useradd: restic now system account
 2021-10-09 20:42:43 +02:00
MichaelEischer
f2a3b3b4a1
Merge pull request #3537 from greatroar/dump-lru 
Check cap instead of len in bloblru
 2021-10-09 20:00:27 +02:00
MichaelEischer
58e8b34633
Merge pull request #3532 from garrmcnu/s3-credentials-config 
s3: Add warning if key ID or secret is empty
 2021-10-09 19:32:32 +02:00
MichaelEischer
a02cea6e83
Merge pull request #3539 from jnoxon/fix-ec2-metadata-v2 
get iam roles working with ec2 metadata v2
 2021-10-07 23:00:27 +02:00
Garry McNulty
708d7a2574 s3: Add warning if key ID or secret is empty 
Also add debug message if no credential types are available.

Closes #2388
 2021-10-06 23:13:40 +01:00
Jeff Noxon
6f4b5ab8d1 get iam roles working with ec2 metadata v2 2021-10-03 20:06:55 -05:00
greatroar
634a9c162d Check cap instead of len in bloblru 
restic dump uses bloblru.Cache to keep buffers alive, but also reuses
evicted buffers. That means large buffers may be used to store small
blobs, causing the cache to think it's using less memory than it
actually does.
 2021-10-03 09:34:17 +02:00
jtagcat
632ca2ef52 docs/example: useradd: restic now system account 
and: use /sbin/nologin
and: use long flags in the useradd command
and: download v0.12.1 instead of v0.9.6
 2021-10-03 00:51:06 +03:00
MichaelEischer
24088f8307
Merge pull request #3528 from greatroar/tree-insert 
internal/restic: Don't allocate in Tree.Insert
 2021-09-27 22:22:03 +02:00
greatroar
c892c0bab9 internal/restic: Don't allocate in Tree.Insert 
name         old time/op    new time/op    delta
BuildTree-8    34.6µs ± 4%     7.0µs ± 3%  -79.68%  (p=0.000 n=18+19)

name         old alloc/op   new alloc/op   delta
BuildTree-8    34.0kB ± 0%     0.9kB ± 0%  -97.37%  (p=0.000 n=20+20)

name         old allocs/op  new allocs/op  delta
BuildTree-8       108 ± 0%         1 ± 0%  -99.07%  (p=0.000 n=20+20)
 2021-09-26 18:08:48 +02:00
MichaelEischer
78dac2fd48
Merge pull request #3523 from greatroar/swift-v2 
Upgrade ncw/swift to v2
 2021-09-24 21:41:36 +02:00
Michael Eischer
5ea8bba1a1 swift: restore context err check for list() 2021-09-24 21:19:46 +02:00
MichaelEischer
a5e103a212
Merge pull request #3522 from greatroar/dump-lru 
Use LRU cache in restic dump
 2021-09-24 20:33:58 +02:00
greatroar
e7ec0453b1 Upgrade ncw/swift to v2 2021-09-24 19:08:37 +02:00
Uli Martens
1ebcb1d097 Add changelog entry for PR #3508 2021-09-24 15:45:09 +02:00
greatroar
fe04d024c7 Use LRU cache in restic dump 2021-09-24 15:45:08 +02:00
Uli Martens
718966a81a Move Blobcache into dedicated internal package 2021-09-24 15:45:00 +02:00
greatroar
4f33eca634 Remove unused Writer arg to internal/dump.writeDump 2021-09-24 15:40:42 +02:00
MichaelEischer
cc110c42e6
Merge pull request #2657 from mansam/add-skip-tls-verification-flag 
Add --insecure-tls flag to disable SSL cert verification
 2021-09-22 21:40:01 +02:00
Sam Lucidi
897d8e662c Add --insecure-tls flag to disable SSL cert verification 
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
 2021-09-21 10:52:40 -04:00
cimnine
4a95af5290 GitHub action for Docker build 2021-09-21 08:23:33 +02:00
cimnine
f28c8bc1c2 Multistage Docker build 2021-09-21 08:23:33 +02:00
MichaelEischer
1827b16ade
Merge pull request #3519 from greatroar/maphash 
Replace siphash by hash/maphash
 2021-09-19 19:46:03 +02:00
greatroar
8b758c78a3 Require Go 1.14 to build 2021-09-19 16:18:19 +02:00
greatroar
8d2996eaaa Replace siphash by hash/maphash 
In Go 1.17.1, maphash has become quite a bit faster than siphash, so we
can drop one third-party dependency. maphash is just an interface to the
standard Go map's hash function, which we already trust for other use
cases.

Benchmark results on linux/amd64, -benchtime=3s:

name                                             old time/op    new time/op    delta
IndexHasUnknown-8                                  50.6ns ±10%    41.0ns ±19%  -18.92%  (p=0.000 n=9+10)
IndexHasKnown-8                                    52.6ns ±12%    41.5ns ±12%  -21.13%  (p=0.000 n=9+10)
IndexMapHash-8                                     3.64µs ± 1%    2.00µs ± 0%  -45.09%  (p=0.000 n=10+9)
IndexAlloc-8                                        700ms ± 1%     601ms ± 6%  -14.18%  (p=0.000 n=8+10)
IndexAllocParallel-8                                205ms ± 5%     192ms ± 8%   -6.18%  (p=0.043 n=10+10)
MasterIndexAlloc-8                                  319ms ± 1%     279ms ± 5%  -12.58%  (p=0.000 n=10+10)
MasterIndexLookupSingleIndex-8                      156ns ± 8%     147ns ± 6%   -5.46%  (p=0.023 n=10+10)
MasterIndexLookupMultipleIndex-8                    150ns ± 7%     142ns ± 8%   -5.69%  (p=0.007 n=10+10)
MasterIndexLookupSingleIndexUnknown-8              74.4ns ± 6%    72.0ns ± 9%     ~     (p=0.175 n=10+9)
MasterIndexLookupMultipleIndexUnknown-8            67.4ns ± 9%    65.5ns ± 7%     ~     (p=0.340 n=9+9)
MasterIndexLookupParallel/known,indices=25-8        461ns ± 2%     445ns ± 2%   -3.49%  (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=25-8      408ns ±11%     378ns ± 5%   -7.22%  (p=0.035 n=10+9)
MasterIndexLookupParallel/known,indices=50-8        479ns ± 1%     437ns ± 4%   -8.82%  (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=50-8      406ns ± 8%     343ns ±15%  -15.44%  (p=0.001 n=10+10)
MasterIndexLookupParallel/known,indices=100-8       480ns ± 1%     455ns ± 5%   -5.15%  (p=0.000 n=8+10)
MasterIndexLookupParallel/unknown,indices=100-8     391ns ±18%     382ns ± 8%     ~     (p=0.315 n=10+10)
MasterIndexLookupBlobSize-8                        71.0ns ± 8%    57.2ns ±11%  -19.36%  (p=0.000 n=9+10)
PackerManager-8                                     254ms ± 1%     254ms ± 1%     ~     (p=0.285 n=15+15)

name                                             old speed      new speed      delta
IndexMapHash-8                                   1.12GB/s ± 1%  2.05GB/s ± 0%  +82.13%  (p=0.000 n=10+9)
PackerManager-8                                   208MB/s ± 1%   207MB/s ± 1%     ~     (p=0.281 n=15+15)

name                                             old alloc/op   new alloc/op   delta
IndexMapHash-8                                      0.00B          0.00B          ~     (all equal)
IndexAlloc-8                                        400MB ± 0%     400MB ± 0%     ~     (p=1.000 n=9+10)
IndexAllocParallel-8                                401MB ± 0%     401MB ± 0%   +0.00%  (p=0.000 n=10+10)
MasterIndexAlloc-8                                  258MB ± 0%     262MB ± 0%   +1.42%  (p=0.000 n=9+10)
PackerManager-8                                    73.1kB ± 0%    73.1kB ± 0%     ~     (p=0.382 n=13+13)

name                                             old allocs/op  new allocs/op  delta
IndexMapHash-8                                       0.00           0.00          ~     (all equal)
IndexAlloc-8                                         907k ± 0%      907k ± 0%   -0.00%  (p=0.000 n=10+10)
IndexAllocParallel-8                                 907k ± 0%      907k ± 0%   +0.00%  (p=0.009 n=10+10)
MasterIndexAlloc-8                                   327k ± 0%      317k ± 0%   -3.06%  (p=0.000 n=10+10)
PackerManager-8                                       744 ± 0%       744 ± 0%     ~     (all equal)
 2021-09-19 16:05:18 +02:00
MichaelEischer
58efe21eca
Merge pull request #3264 from amozoss/upstream-master 
Refactor backup progress
 2021-09-19 14:54:42 +02:00
MichaelEischer
71fcf48533
Merge pull request #2594 from greatroar/concurrent-restore-verify 
Concurrent restore --verify
 2021-09-19 14:52:31 +02:00