KurlesHS/restic
1
0
Fork 0
forked from TrueCloudLab/restic
Code Pull requests Activity
517 commits 10 branches 49 tags 79 MiB
Commit graph

78 commits

Author SHA1 Message Date
Alexander Neumann
4a1cb65645 Merge branch 'replace_arrar' 2015-05-02 14:34:33 +02:00
Florian Weingarten
e19c87fa7d Break Archiver.SaveFile() method down into smaller ones 2015-05-02 01:05:49 -04:00
Alexander Neumann
97619e743c replace github.com/juju/arrar by /errors 2015-05-02 01:29:54 +02:00
Alexander Neumann
0662c14fee Move chunkerBufSize to more appropiate location 2015-05-01 17:40:50 +02:00
Florian Weingarten
0d9360a815 more cleanup 2015-04-29 22:33:34 -04:00
Alexander Neumann
60a0fe8349 Major restructure, bundle blobs 
This commit also breaks the repository format.
 2015-04-29 23:27:32 +02:00
Alexander Neumann
d19b23d4f1 Move Server and Key to new sub-package 2015-04-26 14:46:15 +02:00
Alexander Neumann
434c79354e Merge pull request #143 from fw42/count_errors 
Count errors separately and not as files that are "done"
 2015-04-26 12:05:09 +02:00
Florian Weingarten
61b1a5b1de Count errors separately and not as files that are "done" 2015-04-26 01:54:35 +00:00
Florian Weingarten
c947623707 Fix progress off-by-1 2015-04-25 15:40:42 -04:00
Florian Weingarten
262e5a8b4a O_NOATIME on Linux 2015-04-25 00:45:20 +00:00
Alexander Neumann
b2307cafa2 Seed chunker with random per-repository polynomial 2015-04-06 00:22:19 +02:00
Alexander Neumann
24b14e21cc Remove pools for nodes and IDs 
This removes the allocation pools for nodes and IDs. I feel they aren't
really needed. Benchmarks:

    benchmark                         old ns/op      new ns/op      delta
    BenchmarkChunkEncrypt             197890867      198616293      +0.37%
    BenchmarkChunkEncryptParallel     196127004      198819818      +1.37%
    BenchmarkArchiveDirectory         1098848419     1087237723     -1.06%
    BenchmarkPreload                  30464455       29910239       -1.82%
    BenchmarkLoadTree                 3265092        3088543        -5.41%
    BenchmarkEncryptWriter            37213511       37134683       -0.21%
    BenchmarkEncrypt                  36037879       36166546       +0.36%
    BenchmarkDecryptReader            38165659       38556734       +1.02%
    BenchmarkEncryptDecryptReader     77027044       77194987       +0.22%
    BenchmarkDecrypt                  36017602       35937888       -0.22%
    BenchmarkSaveJSON                 47906          50270          +4.93%
    BenchmarkSaveFrom                 49775973       50520969       +1.50%
    BenchmarkLoadJSONID               105290245      107281849      +1.89%
    BenchmarkChunkerWithSHA256        151501430      148264078      -2.14%
    BenchmarkChunkerWithMD5           93606346       94036392       +0.46%
    BenchmarkChunker                  74285431       75933882       +2.22%
    BenchmarkPipelineWalker           387689         346467         -10.63%

    benchmark                         old MB/s     new MB/s     speedup
    BenchmarkChunkEncrypt             52.99        52.79        1.00x
    BenchmarkChunkEncryptParallel     53.46        52.74        0.99x
    BenchmarkEncryptWriter            225.42       225.90       1.00x
    BenchmarkEncrypt                  232.77       231.94       1.00x
    BenchmarkDecryptReader            219.79       217.57       0.99x
    BenchmarkEncryptDecryptReader     108.90       108.67       1.00x
    BenchmarkDecrypt                  232.90       233.42       1.00x
    BenchmarkSaveFrom                 84.26        83.02        0.99x
    BenchmarkChunkerWithSHA256        69.21        70.72        1.02x
    BenchmarkChunkerWithMD5           112.02       111.51       1.00x
    BenchmarkChunker                  141.15       138.09       0.98x

    benchmark                         old allocs     new allocs     delta
    BenchmarkChunkEncrypt             110            110            +0.00%
    BenchmarkChunkEncryptParallel     100            100            +0.00%
    BenchmarkArchiveDirectory         475591         476635         +0.22%
    BenchmarkPreload                  28059          24182          -13.82%
    BenchmarkLoadTree                 3124           2889           -7.52%
    BenchmarkEncryptWriter            19             19             +0.00%
    BenchmarkEncrypt                  13             13             +0.00%
    BenchmarkDecryptReader            16             15             -6.25%
    BenchmarkEncryptDecryptReader     39             39             +0.00%
    BenchmarkDecrypt                  11             11             +0.00%
    BenchmarkSaveJSON                 74             74             +0.00%
    BenchmarkSaveFrom                 109            112            +2.75%
    BenchmarkLoadJSONID               103630         97849          -5.58%
    BenchmarkChunkerWithSHA256        13             13             +0.00%
    BenchmarkChunkerWithMD5           12             12             +0.00%
    BenchmarkChunker                  6              6              +0.00%
    BenchmarkPipelineWalker           212            165            -22.17%

    benchmark                         old bytes     new bytes     delta
    BenchmarkChunkEncrypt             64697         64697         +0.00%
    BenchmarkChunkEncryptParallel     64681         64681         +0.00%
    BenchmarkArchiveDirectory         193385504     193790864     +0.21%
    BenchmarkPreload                  4064701       3942000       -3.02%
    BenchmarkLoadTree                 344954        325396        -5.67%
    BenchmarkEncryptWriter            12793         12793         +0.00%
    BenchmarkEncrypt                  1950          1950          +0.00%
    BenchmarkDecryptReader            3120          2774          -11.09%
    BenchmarkEncryptDecryptReader     1528036       1528036       +0.00%
    BenchmarkDecrypt                  1919          1919          +0.00%
    BenchmarkSaveJSON                 5524          5524          +0.00%
    BenchmarkSaveFrom                 31353         40804         +30.14%
    BenchmarkLoadJSONID               12872020      16010968      +24.39%
    BenchmarkChunkerWithSHA256        26821         26821         +0.00%
    BenchmarkChunkerWithMD5           13554         13554         +0.00%
    BenchmarkChunker                  13458         13458         +0.00%
    BenchmarkPipelineWalker           58584         55560         -5.16%
 2015-03-28 16:51:37 +01:00
Alexander Neumann
f157f775da Add checks for Server.Load(), use Blob for load 2015-03-28 15:07:08 +01:00
Alexander Neumann
5e69788eac Refactor backends 2015-03-28 14:36:50 +01:00
Alexander Neumann
cfa2229bc0 Refactor cache refresh for blobs, add progress 2015-03-22 15:09:50 +01:00
Alexander Neumann
aca0692ee6 Add more error reporting 2015-03-21 14:43:33 +01:00
Alexander Neumann
71a57537ef Report errors, ignore files and continue. Closes #108 2015-03-15 15:48:05 +01:00
Alexander Neumann
3d768e39ea Add more error reporting 2015-03-15 12:20:30 +01:00
Alexander Neumann
3c92c7e689 Recreate blob cache if missing (closes #104) 2015-03-14 18:15:47 +01:00
Alexander Neumann
e1fc17aeb1 Implement per-repository cache 2015-03-14 12:16:28 +01:00
Alexander Neumann
7b54c21726 Store ID in Snapshot 2015-03-10 14:17:52 +01:00
Alexander Neumann
2166367039 Fix comparison for new files with '.' 2015-03-10 14:17:50 +01:00
Alexander Neumann
14298fe232 Cache blobs for each snapshot 2015-03-09 23:50:32 +01:00
Alexander Neumann
ab8b97c4ba Remove code duplication 
The top-level tree is now handled by the DirWorkers
 2015-03-08 21:06:51 +01:00
Alexander Neumann
b3cdd3ead9 Remove unused code 2015-03-08 14:22:35 +01:00
Alexander Neumann
ba892e1ec2 Refactor backup pipeline 2015-03-08 14:16:01 +01:00
Alexander Neumann
f167366be5 Add walk for trees, restructure 2015-03-02 19:58:06 +01:00
Alexander Neumann
c8be54564f Add local cache 2015-03-02 10:04:01 +01:00
Alexander Neumann
a59b0ec1f6 Add progress report for loading blobs 2015-02-21 15:32:48 +01:00
Alexander Neumann
58cded6b75 Refactor progress reporting 2015-02-21 14:25:06 +01:00
Alexander Neumann
97b963ef1d Report files and dirs 2015-02-21 14:05:02 +01:00
Alexander Neumann
a0fea201d9 Reenable Preloading 2015-02-18 23:10:59 +01:00
Alexander Neumann
f8e47181b0 Add more debug for preloading 2015-02-17 23:38:40 +01:00
Alexander Neumann
677a5038c5 Increase concurrency for loading blobs 2015-02-17 23:15:08 +01:00
Alexander Neumann
4bb7f2f2ed Add Preloader for blobs 2015-02-17 22:39:44 +01:00
Alexander Neumann
4fd8222da6 Remove unneeded buffers from Archiver.SaveFile() 2015-02-16 23:47:32 +01:00
Alexander Neumann
06ed5c12b8 Remove Compress() and Uncompress() from backend 2015-02-16 00:30:51 +01:00
Alexander Neumann
f8f8107d55 wip 2015-02-15 15:17:24 +01:00
Alexander Neumann
bbb3145f11 Add chunker pool 2015-02-09 23:38:50 +01:00
Alexander Neumann
bda33e612c Implement streaming chunker using io.Reader 2015-02-09 00:28:54 +01:00
Alexander Neumann
25a214809b Refactor debug into debug module 2015-01-17 16:32:13 +01:00
Alexander Neumann
af43fb3dda Fix handling vanished files 2015-01-14 17:48:23 +01:00
Alexander Neumann
4e54c6af2c Fix calls to debug() 2015-01-14 17:48:10 +01:00
Alexander Neumann
48751e1935 Introduce debug tags and debug breaks 
The environment variable DEBUG_TAGS can be used to control what is
printed on stderr. Example:

    DEBUG_TAGS="+all,-Archiver.*" ./restic backup foo

The variable DEBUG_BREAK can be used to set a breakpoint (implemented
using SIGSTOP):

    DEBUG_BREAK=Archiver.Snapshot ./restic backup foo
 2015-01-14 16:34:30 +01:00
Alexander Neumann
203a911de9 Refactor repository structure 
Merge Map data type into Tree.
 2015-01-14 16:34:30 +01:00
Alexander Neumann
cdf3336e7a Improve error message for newer files 2015-01-08 20:49:32 +01:00
Alexander Neumann
33bcf31bae Fix incremental backup 
Copying blobs (for unchanged files and subtrees) from old BlobList to
new BlobList was missing
 2015-01-05 21:40:43 +01:00
Alexander Neumann
6eb969a492 Update modified files, store error message 2015-01-04 23:25:13 +01:00
Alexander Neumann
e543f5926c Refactor Archiver and backup command 
Improve incremental backup by first scanning the tree, loading the old
tree and afterwards comparing both trees in memory.
 2015-01-04 22:39:30 +01:00