Rached Ben Mustapha
83bbf21f1a
checker: Validate pack checksums before unpacking
...
This avoids reading a possibly invalid size at the end of a corrupted pack
2016-02-04 22:55:39 +01:00
Alexander Neumann
4f1f03cdb9
Move testing for known blobs to Archiver
...
This removes the list of in-flight blobs from the master index and
instead keeps a list of "known" blobs in the Archiver. "known" here
means: either already processed, or included in an index. This property
is tested atomically, when the blob is not in the list of "known" blobs,
it is added to the list and the caller is responsible to make this
happen (i.e. save the blob).
2016-02-01 23:50:56 +01:00
Alexander Neumann
280d580ae2
checker: Use Load() instead of GetReader()
2016-01-24 00:42:04 +01:00
Alexander Neumann
0b50f9e02c
Move MemoryBackend to backend/mem
2016-01-23 19:50:11 +01:00
Alexander Neumann
8b7bf8691d
backend: Remove Get()
...
This is the first commit that removes the (redundant) Get() method of
the backend interface. Get(x, y) is equivalent to GetReader(x, y, 0, 0).
2016-01-23 13:13:05 +01:00
Alexander Neumann
f53008d916
Allow saving duplicate blobs in the repacker
...
This adds code to the master index to allow saving duplicate blobs
within the repacker. In this mode, only the list of currently in flight
blobs is consulted, and not the index. This correct because while
repacking, a unique list of blobs is saved again to the index.
2016-01-17 21:14:55 +01:00
Alexander Neumann
d1ca986f55
Fix tests
2015-12-06 17:38:51 +01:00
Alexander Neumann
3ac1d0e4d1
add progress
2015-12-06 17:29:31 +01:00
Alexander Neumann
0e66a66bce
read packs concurrently
2015-12-06 17:09:06 +01:00
Alexander Neumann
43a23f91a6
checker: add function to read and verify all data
2015-12-02 22:40:36 +01:00
Alexander Neumann
6c54d3fa82
index: also mark old index as final on decode
2015-11-08 22:21:29 +01:00
Alexander Neumann
c59b12c939
Show a hint whech the checker finds an old index
2015-11-08 21:50:48 +01:00
Alexander Neumann
0222b1701e
Remvoe automatic index conversion
2015-11-08 21:35:48 +01:00
Alexander Neumann
db41102bfa
Finalize repacker
2015-11-02 19:28:30 +01:00
Alexander Neumann
266bc05edc
Add mostly ready repacker
2015-11-02 18:52:13 +01:00
Alexander Neumann
51aff3ca57
Add FindBlobsForPacks()
2015-11-02 18:52:13 +01:00
Alexander Neumann
484331cd8d
Add repacker
2015-11-02 18:52:13 +01:00
Alexander Neumann
96ecc26507
Let the checker return a list of hints along with errors
2015-10-25 16:26:50 +01:00
Alexander Neumann
91e1929b52
checker: test for packs in multiple indexes
2015-10-25 16:00:06 +01:00
Alexander Neumann
86fcd170f6
Add and use MasterIndex
2015-10-12 22:34:12 +02:00
Alexander Neumann
eb73182fcf
Rework index decode and handling old format
2015-10-12 20:53:07 +02:00
Alexander Neumann
1020e9c3af
Check for data blobs with null ID, improve errors
2015-10-11 20:58:57 +02:00
Alexander Neumann
7db2369081
Shorten error message for tree errors
2015-10-11 20:22:52 +02:00
Alexander Neumann
db85ab8aa0
Use the correct channel for sending errors
2015-10-11 19:13:45 +02:00
Alexander Neumann
86c8328f62
Handle null subtree IDs
2015-10-11 19:13:35 +02:00
Alexander Neumann
72fcd00859
Check subtrees with null ID
2015-10-11 18:46:26 +02:00
Alexander Neumann
8a7873ee3a
Handle invalid subtree IDs
2015-10-11 18:45:16 +02:00
Alexander Neumann
a0bad1695c
Remove comment
2015-09-05 18:41:58 +02:00
Alexander Neumann
9753c37e31
Remove dead code
...
This removes dead code (functions that aren't called) detected with
`deadcode`.
2015-07-26 14:56:34 +02:00
Alexander Neumann
5cdcc99eba
Use array instead of hash for backend.ID
...
Since backend.ID is always a slice of constant length, use an array
instead of a slice. Mostly, arrays behave as slices, except that an
array cannot be nil, so use `*backend.ID` insteaf of `backend.ID` in
places where the absence of an ID is possible (e.g. for the Subtree of a
Node, which may not present when the node is a file node).
This change allows to directly use backend.ID as the the key for a map,
so that arbitrary data structures (e.g. a Set implemented as a
map[backend.ID]struct{}) can easily be formed.
2015-07-25 18:01:57 +02:00
Alexander Neumann
d36f07c6eb
checker: add option to remove orphaned packs
2015-07-12 17:14:10 +02:00
Alexander Neumann
5108d91bc7
checker: check trees and blobs in parallel
2015-07-12 16:50:12 +02:00
Alexander Neumann
af02c323cd
checker: use channel of error instead of slice
2015-07-12 01:46:10 +02:00
Alexander Neumann
7e6174126f
checker: run Packs() in parallel
2015-07-12 00:25:42 +02:00
Alexander Neumann
04f172b38d
checker: Add checker and tests
2015-07-11 16:27:41 +02:00
Alexander Neumann
54c4c29a89
Add checker and command 'check' to replace 'fsck'
2015-07-11 16:27:38 +02:00
