Commit graph

4702 commits

Author SHA1 Message Date
Alexander Neumann
734ae7fcb8 Add test for corner case
It was observed that a restic repository still contained overlapping
indexes after `rebuild-index` has been called. This is caused by
instantly forgetting that blobs have already been saved once a full
index has been written during index rebuilding.

This commit adds a (failing) test that shows the behaviour.
2015-10-25 21:51:57 +01:00
Alexander Neumann
7b8e42a763 Silence rebuild-index tests 2015-10-25 21:51:46 +01:00
Alexander Neumann
566fb22bcf Merge pull request #324 from restic/fix-index-grow
Fix really large index and memory exhaustion
2015-10-25 18:58:31 +01:00
Alexander Neumann
b88ccb4f1b Fix Unpacker test 2015-10-25 18:21:48 +01:00
Alexander Neumann
efbce9f0fa rebuild-index: handle not yet indexed packs 2015-10-25 18:07:51 +01:00
Alexander Neumann
88849c06a6 rebuild-index: Refactor a bit 2015-10-25 17:53:02 +01:00
Alexander Neumann
5d617edbbf local/sftp backend: Do not seek if offset is 0 2015-10-25 17:51:26 +01:00
Alexander Neumann
6aed9f268b Add command rebuild-index 2015-10-25 17:24:52 +01:00
Alexander Neumann
9074c923ea index: add AddToSupersedes() 2015-10-25 17:06:56 +01:00
Alexander Neumann
1365495599 debug: remove extra space between filename and line 2015-10-25 17:06:20 +01:00
Alexander Neumann
461d54e43c Refactor repository.SaveIndex() 2015-10-25 17:05:54 +01:00
Alexander Neumann
96ecc26507 Let the checker return a list of hints along with errors 2015-10-25 16:26:50 +01:00
Alexander Neumann
91e1929b52 checker: test for packs in multiple indexes 2015-10-25 16:00:06 +01:00
Alexander Neumann
04614c7527 Add test for packs in duplicate indexes 2015-10-25 15:35:33 +01:00
Alexander Neumann
f7ff5b766c Mark written indexes as finalized 2015-10-25 15:35:18 +01:00
Alexander Neumann
d9f9b77d68 Add Index.Packs() and IDSet.Equals() 2015-10-25 15:28:01 +01:00
Alexander Neumann
4b1a2caea7 Allow overwriting the IndexFull function for tests 2015-10-25 15:05:22 +01:00
Alexander Neumann
af0d6f58b9 Remove unneeded pointer to pack id 2015-10-25 14:35:08 +01:00
Alexander Neumann
2710d6399a Cleanup index code
The selectFn wasn't used any more, so remove it from generatePackList().
2015-10-25 14:26:04 +01:00
Alexander Neumann
650eab6a0e Fix typo in dump usage 2015-10-25 13:19:35 +01:00
Alexander Neumann
5de36dfdf0 Merge pull request #310 from restic/resume-backups
resume interrupted backups
2015-10-14 21:53:25 +02:00
Alexander Neumann
1dd731fdb8 Handle concurrent access to the inFlight list 2015-10-14 20:50:54 +02:00
Alexander Neumann
6fa4be5af2 Regularly save intermediate indexes 2015-10-12 23:59:17 +02:00
Alexander Neumann
941b7025b6 Delete Index.Remove() 2015-10-12 22:51:11 +02:00
Alexander Neumann
4b2a4b03ec Remove Index.StoreInProgress() 2015-10-12 22:49:31 +02:00
Alexander Neumann
7ab9915859 Fix 'dump' command 2015-10-12 22:42:31 +02:00
Alexander Neumann
86fcd170f6 Add and use MasterIndex 2015-10-12 22:34:12 +02:00
Alexander Neumann
64fa89d406 Add error checks 2015-10-12 22:07:56 +02:00
Alexander Neumann
eb73182fcf Rework index decode and handling old format 2015-10-12 20:53:07 +02:00
Alexander Neumann
356bb62243 Add CreateEncryptedBlob and GetDecryptReader 2015-10-12 20:53:07 +02:00
Alexander Neumann
de2c20be84 Dump individual indexes 2015-10-12 20:53:07 +02:00
Alexander Neumann
96f2165067 Allow loading index with old format 2015-10-12 20:53:07 +02:00
Alexander Neumann
7944e8e323 Update index format 2015-10-12 20:53:07 +02:00
Alexander Neumann
5c39abfe53 Merge pull request #319 from restic/fix-311
Handle null subtree IDs
2015-10-11 21:48:52 +02:00
Alexander Neumann
1020e9c3af Check for data blobs with null ID, improve errors 2015-10-11 20:58:57 +02:00
Alexander Neumann
cc7acba02b Return the original backend ID on duplicate entries 2015-10-11 20:45:50 +02:00
Alexander Neumann
f188cf81dc Add more panic() calls for invalid conditions 2015-10-11 20:45:42 +02:00
Alexander Neumann
7db2369081 Shorten error message for tree errors 2015-10-11 20:22:52 +02:00
Alexander Neumann
db85ab8aa0 Use the correct channel for sending errors 2015-10-11 19:13:45 +02:00
Alexander Neumann
86c8328f62 Handle null subtree IDs 2015-10-11 19:13:35 +02:00
Alexander Neumann
72fcd00859 Check subtrees with null ID 2015-10-11 18:46:26 +02:00
Alexander Neumann
8a7873ee3a Handle invalid subtree IDs 2015-10-11 18:45:16 +02:00
Alexander Neumann
e738d35c4e Merge pull request #309 from restic/update-ci-go-version
Update Go 1.4 to version 1.4.3
2015-09-27 20:20:35 +02:00
Alexander Neumann
6ddda5fc5e appveyor: remove old Go installation 2015-09-27 18:34:11 +02:00
Alexander Neumann
7291342723 Install current version of Go
This is inspired by the appveyor.yaml from the go-plus project:
https://github.com/joefitzgerald/go-plus/blob/master/appveyor.yml
2015-09-27 18:02:44 +02:00
Alexander Neumann
70a6233b94 Install the 'cover' tool 2015-09-27 17:58:13 +02:00
Alexander Neumann
749ca28534 Update Go 1.4 to version 1.4.3 2015-09-27 17:22:12 +02:00
Alexander Neumann
321c2e6a47 Merge pull request #308 from episource/fix/restic_cr292_unreferenced_pack
fix:restic#292 Prevent concurrent processing of the same blob
2015-09-27 17:18:28 +02:00
Philipp Serr
7b11660f4f Prevent concurrent processing of same blob
... by first adding a preliminary index entry and making this fail if
an index entry for the same blob already exists.

A preliminary index entry is characterized by not yet being associated
with a pack. Until now, these entries where added to the index just
like final index entries using index.Store, which silently overwrites
existing index entries.

This commit adds a new method index.StoreInProgress which refuses to
overwrite existing index entries and allows for creating preliminary
index entries only. The existing method index.Store has not been
changed and continues to silently overwrite existing index entries.
This distinction is important, as otherwise, it would be impossible to
update a preliminary index entry after the blob has been written to a
pack.

Resolves: restic#292
2015-09-27 16:56:49 +02:00
Alexander Neumann
4fb46faae7 Use Go 1.5.1 for travis tests 2015-09-12 22:15:09 +02:00