george.bartolomey/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
637 commits 7 branches 53 tags 45 MiB
Commit graph

172 commits

Author SHA1 Message Date
Leonard Lyubich
91bea44a1a [#245] blobovnicza: Fix initializing dimensional buckets 
In previous implementation Blobovnicza could incorrectly initialize
dimensional buckets: if SmallSizeLimit = 2 ^ X + Y && Y < 2 ^ X, then
largest dimensional bucket was [2 ^ (X - 1) : 2 ^ X]. This was caused by an
incorrect condition for stopping the iterator along the dimensional
boundaries.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-14 12:48:59 +03:00
Leonard Lyubich
4f5d5c7e45 Write system path to BoltDB on blobovnicza closing 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
63bc3aab82 [#242] engine: Do not iterate over shards if cid is not set 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
511b5f782a [#242] engine: Support ContainerID argument in shard selects 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
f585772746 [#242] shard: Support ContainerID argument in metabase selects 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
b97f818323 [#242] metabase: Make ContainerID a mandatory parameter for Select 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
9dd83bdf0d [#235] blobstor: Return object.ErrRangeOutOfBounds from shallow dir 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
41b9fa5b45 [#235] engine: Return OutOfBounds error 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
1d23483828 [#235] services/object: Implement new GetRange algorithm 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
91d8e0a4de [#238] engine: Support raw flag in Head method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
da42ee3a26 [#238] engine: Stop shard iteration at split info error in Head 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
69dee6ec6d [#238] shard: Support raw flag in Head method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
bf7a390304 [#238] metabase: Support raw flag in Get method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
c267a08886 [#236] engine: Stop shard iteration at split info error in GetRange 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
590745204c [#237] metabase: Structure parameters and results of all operations 
All parameters and resulting values of all metabase operations are
structured in new types. The most popular scenarios for using operations are
moved to auxiliary functions.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
a875d80491 Remove outdated code of metabase and localstore 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
26f03c6301 [#232] Fix SplitInfo update in existed virtual object 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
ee1e3931a5 [#231] engine: Add List method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
a7087faa1a [#231] shard: Add List method to return all available objects 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
a1cb48f800 [#231] metabase: Add container lister 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
3a37eda410 [#230] engine: Stop on first SplitInfoError in Get method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
a488d202e1 [#230] engine: Rename WithAddress to WithAddresses for delete operations 
Delete works with multiple addresses at once so it is convenient to
show it in parameter setter function.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
7c97bd1360 [#230] shard: Add test for getting SplitInfo 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
bf9e938a3b [#230] metabase: Return SplitInfoError in Exist method 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
e2de95e3f6 [#230] metabase: Index split info structures 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
eaae5a5dd7 [#229] blobovnicza: Store objects in a binary format 
In previous implementation Blobovnicza's stored objects in protocol format
which did not allow working with externally compressed objects. To achieve
this goal, operations Get and Put no longer work with the structure of the
object, but only with abstract binary data. Operation GetRange has become
incorrect in its original purpose to receive the payload range. In this
regard, BlobStor receives the payload range of the object through Get
operation. In the future either Blobovnicza will learn to compress objects
by itself, or the GetRange operation will be eliminated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
3d77fdb347 [#222] Set write cache specific config on application level 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
af2719689d [#222] Do not compress data in blobovniczas 
Blobovnicza returns object, so we can't put compressed
data there. Compressed data won't be deserialized correctly.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
6bec50963b [#222] Add tests for shards 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
ce92df3543 [#222] Return nil in metabase.IsSmall if object is big 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
862f9a3f1a [#222] Put small objects in blobovnicza in blobStor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
21708d5408 [#222] Add Exists method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
f470022594 [#222] Update Select method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
25486194b9 [#222] Update Head method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
67d4b0db12 [#222] Move ErrAlreadyRemoved to core package 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
6a3e3e5642 [#222] Update Range method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
c9b1c6603e [#222] Update Get method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
f3e96f8c19 [#222] Update Delete method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
aa8dd18b07 [#222] Update Put method in storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
f45b7a048d [#222] Rename WithTreeRoot.. to WithRoot.. in blobstor opts 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
d4e48b28b1 [#222] Add Inhume method to storage engine 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
92c95150cd [#222] Use write cache options in storage app 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
69ba295077 [#222] Add ToMoveIt method in shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
ccd8760fbc [#222] Update Exists method in shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
97a2e26b71 [#222] Add Head method in shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
244e244c0e [#222] Add GetRange method in shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
73505fe582 [#222] Update Get method in shard 
Now `Get`, `Head` and `GetRange` will be
separate methods on shard.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
31371771e8 [#222] Update Delete method in shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
5fc54386b0 [#222] Update Put method in shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Alex Vanin
72f8ca1afd [#222] Add Inhume method for shard 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00