KurlesHS/restic
1
0
Fork 0
forked from TrueCloudLab/restic
Code Pull requests Activity
1253 commits 10 branches 49 tags 79 MiB
Commit graph

86 commits

Author SHA1 Message Date
Alexander Neumann
4f1f03cdb9 Move testing for known blobs to Archiver 
This removes the list of in-flight blobs from the master index and
instead keeps a list of "known" blobs in the Archiver. "known" here
means: either already processed, or included in an index. This property
is tested atomically, when the blob is not in the list of "known" blobs,
it is added to the list and the caller is responsible to make this
happen (i.e. save the blob).
 2016-02-01 23:50:56 +01:00
Alexander Neumann
382c766983 Move test for #405: Test Archiver instead of Repo 2016-02-01 23:50:41 +01:00
Alexander Neumann
f5f6e9cf37 Add test to reproduce #405 2016-02-01 23:35:01 +01:00
Alexander Neumann
f2371db2a9 repository: remove decryptReadCloser 2016-01-27 22:47:09 +01:00
Alexander Neumann
ea29ad6f96 Remove last ocurrence of Create() 2016-01-24 19:30:14 +01:00
Alexander Neumann
ac2fe4e04f Remove BlobWriter 2016-01-24 18:53:39 +01:00
Alexander Neumann
cfdd3a853d Remove usage of CreateEncryptedBlob() 2016-01-24 18:52:11 +01:00
Alexander Neumann
01e40e62bf repo: Use Save() instead of Create() 2016-01-24 18:50:41 +01:00
Alexander Neumann
fe565e17c3 Key: Use Save() instead of Create() 2016-01-24 17:52:44 +01:00
Alexander Neumann
782a1bf7b0 repository: remove GetDecryptReader() 2016-01-24 00:12:17 +01:00
Alexander Neumann
3191778d33 repository: Use Load() instead of GetReader() 2016-01-24 00:12:09 +01:00
Alexander Neumann
9bfa633187 repository/key: Use Load() instead of GetReader() 2016-01-23 23:48:19 +01:00
Alexander Neumann
10b03eee27 Add comment 2016-01-23 23:27:40 +01:00
Alexander Neumann
8b7bf8691d backend: Remove Get() 
This is the first commit that removes the (redundant) Get() method of
the backend interface. Get(x, y) is equivalent to GetReader(x, y, 0, 0).
 2016-01-23 13:13:05 +01:00
Alexander Neumann
f53008d916 Allow saving duplicate blobs in the repacker 
This adds code to the master index to allow saving duplicate blobs
within the repacker. In this mode, only the list of currently in flight
blobs is consulted, and not the index. This correct because while
repacking, a unique list of blobs is saved again to the index.
 2016-01-17 21:14:55 +01:00
Alexander Neumann
34c1056efc Merge pull request #358 from episource/iss358_pack_not_referened_add_test 
Closes #365
Closes #358
 2016-01-17 20:07:56 +01:00
Alexander Neumann
6a56d5b87b Repo: Add more debug 2016-01-17 18:48:05 +01:00
Philipp Serr
0fde09a866 Lock MasterIndex and InFlight store together 
fixes: #358
 2015-12-28 18:40:43 +01:00
Alexander Neumann
8209bb309b split out decryptReader and packerManager 2015-11-29 14:29:59 +01:00
Alexander Neumann
d4b873ca76 Repo: add documentation 2015-11-29 14:29:57 +01:00
Alexander Neumann
5b601f00b1 Add error checking 2015-11-29 14:25:57 +01:00
Alexander Neumann
567de35df4 Save new packs in index atomically 
This commit fixes a situation reported by a user where two indexes
contained information about the same pack without overlap, e.g.:

Index 3e6a32 contained:

    {
      "id": "c02e3b",
      "blobs": [
        {
          "id": "8114b1",
          "type": "data",
          "offset": 0,
          "length": 530107
        }
      ]
    }

And index 62da5f contained:

    {
      "id": "c02e3b",
      "blobs": [
        {
          "id": "e344f8",
          "type": "data",
          "offset": 1975848,
          "length": 3426468
        },
        {
          "id": "939ed9",
          "type": "data",
          "offset": 530107,
          "length": 1445741
        }
      ]
    }

This commit adds all blobs in a pack in one atomic operation so that
intermediate such as these do not happen.
 2015-11-20 22:56:56 +01:00
Alexander Neumann
96061d2a2f Fix debug log message 2015-11-13 23:47:53 +01:00
Alexander Neumann
5776b8f01c remove ConvertIndex 2015-11-08 22:27:13 +01:00
Alexander Neumann
6c54d3fa82 index: also mark old index as final on decode 2015-11-08 22:21:29 +01:00
Alexander Neumann
db41102bfa Finalize repacker 2015-11-02 19:28:30 +01:00
Alexander Neumann
1fc0d78913 Refactor Index.Store() to take a PackedBlob 2015-11-02 19:07:03 +01:00
Alexander Neumann
f3f84b1544 Add ID handling for index 2015-11-02 18:52:13 +01:00
Alexander Neumann
60a34087c9 Move LoadIndexWithDecoder to index.go 2015-11-02 18:52:13 +01:00
Alexander Neumann
89a77ab2f9 Add Index.ListPack() 2015-11-02 18:52:13 +01:00
Alexander Neumann
fccde030d5 Refactor Index.Lookup() to return struct PackedBlob 2015-10-31 14:47:42 +01:00
Alexander Neumann
7711fcda69 use new index format for repository tests 2015-10-27 20:51:55 +01:00
Alexander Neumann
7717ea5cca Add benchmark for LoadJSONPack 2015-10-27 20:51:55 +01:00
Alexander Neumann
88849c06a6 rebuild-index: Refactor a bit 2015-10-25 17:53:02 +01:00
Alexander Neumann
9074c923ea index: add AddToSupersedes() 2015-10-25 17:06:56 +01:00
Alexander Neumann
461d54e43c Refactor repository.SaveIndex() 2015-10-25 17:05:54 +01:00
Alexander Neumann
04614c7527 Add test for packs in duplicate indexes 2015-10-25 15:35:33 +01:00
Alexander Neumann
f7ff5b766c Mark written indexes as finalized 2015-10-25 15:35:18 +01:00
Alexander Neumann
d9f9b77d68 Add Index.Packs() and IDSet.Equals() 2015-10-25 15:28:01 +01:00
Alexander Neumann
4b1a2caea7 Allow overwriting the IndexFull function for tests 2015-10-25 15:05:22 +01:00
Alexander Neumann
af0d6f58b9 Remove unneeded pointer to pack id 2015-10-25 14:35:08 +01:00
Alexander Neumann
2710d6399a Cleanup index code 
The selectFn wasn't used any more, so remove it from generatePackList().
 2015-10-25 14:26:04 +01:00
Alexander Neumann
1dd731fdb8 Handle concurrent access to the inFlight list 2015-10-14 20:50:54 +02:00
Alexander Neumann
6fa4be5af2 Regularly save intermediate indexes 2015-10-12 23:59:17 +02:00
Alexander Neumann
941b7025b6 Delete Index.Remove() 2015-10-12 22:51:11 +02:00
Alexander Neumann
4b2a4b03ec Remove Index.StoreInProgress() 2015-10-12 22:49:31 +02:00
Alexander Neumann
86fcd170f6 Add and use MasterIndex 2015-10-12 22:34:12 +02:00
Alexander Neumann
64fa89d406 Add error checks 2015-10-12 22:07:56 +02:00
Alexander Neumann
eb73182fcf Rework index decode and handling old format 2015-10-12 20:53:07 +02:00
Alexander Neumann
356bb62243 Add CreateEncryptedBlob and GetDecryptReader 2015-10-12 20:53:07 +02:00