Alex Vanin
e2de95e3f6
[ #230 ] metabase: Index split info structures
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
eaae5a5dd7
[ #229 ] blobovnicza: Store objects in a binary format
...
In previous implementation Blobovnicza's stored objects in protocol format
which did not allow working with externally compressed objects. To achieve
this goal, operations Get and Put no longer work with the structure of the
object, but only with abstract binary data. Operation GetRange has become
incorrect in its original purpose to receive the payload range. In this
regard, BlobStor receives the payload range of the object through Get
operation. In the future either Blobovnicza will learn to compress objects
by itself, or the GetRange operation will be eliminated.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
3d77fdb347
[ #222 ] Set write cache specific config on application level
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
af2719689d
[ #222 ] Do not compress data in blobovniczas
...
Blobovnicza returns object, so we can't put compressed
data there. Compressed data won't be deserialized correctly.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
4114028a63
[ #222 ] Add shard amount check
...
There should be at least one correctly set up shard in node config.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
6bec50963b
[ #222 ] Add tests for shards
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
ce92df3543
[ #222 ] Return nil in metabase.IsSmall if object is big
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
862f9a3f1a
[ #222 ] Put small objects in blobovnicza in blobStor
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
351e4b4592
[ #222 ] Support Inhume and Delete in object service
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
21708d5408
[ #222 ] Add Exists method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
f470022594
[ #222 ] Update Select method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
25486194b9
[ #222 ] Update Head method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
67d4b0db12
[ #222 ] Move ErrAlreadyRemoved to core package
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
6a3e3e5642
[ #222 ] Update Range method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
c9b1c6603e
[ #222 ] Update Get method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
f3e96f8c19
[ #222 ] Update Delete method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
aa8dd18b07
[ #222 ] Update Put method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
f45b7a048d
[ #222 ] Rename WithTreeRoot.. to WithRoot.. in blobstor opts
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
d4e48b28b1
[ #222 ] Add Inhume method to storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
92c95150cd
[ #222 ] Use write cache options in storage app
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
69ba295077
[ #222 ] Add ToMoveIt method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
ccd8760fbc
[ #222 ] Update Exists method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
97a2e26b71
[ #222 ] Add Head method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
244e244c0e
[ #222 ] Add GetRange method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
73505fe582
[ #222 ] Update Get method in shard
...
Now `Get`, `Head` and `GetRange` will be
separate methods on shard.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
31371771e8
[ #222 ] Update Delete method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
5fc54386b0
[ #222 ] Update Put method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
72f8ca1afd
[ #222 ] Add Inhume method for shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
1b76458684
[ #222 ] Add writeCache instance to shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2eb83624cf
[ #220 ] cmd/neofs-node: Configure Shard's write cache usage
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
cbdcfb9159
[ #220 ] shard: Enable write cache through option
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
47d2239332
[ #220 ] localstorage: Replace basic errors to core library
...
Replace ErrNotFound and ErrRangeOutOfBounds to core/object package in order
to share them across the libraries.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
7512a5ba18
[ #220 ] blobstor: Implement "small" operations
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
7f20e6f9db
[ #220 ] blobstor: Implement GetRange on blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
ef095d4871
[ #220 ] blobovnicza: Implement GetRange operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
cd2228cd99
[ #220 ] cmd/neofs-node: Support new configuration of storage components
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
e758c246b7
[ #220 ] localstorage: Implement Open/Init/Close methods
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2963473c08
[ #220 ] localstorage: Implement hrw.Hasher over Shard structure
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
88c1584e6a
[ #220 ] shard: Implement Open/Init/Close methods
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
d30454a572
[ #220 ] shard: Use new metabase implementation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
d8d38d3476
[ #220 ] blobstor: Add blobovnicza tree structure to BlobStor
...
Add blobovnicza instance to BlobStor structure. Create blobovnicza tree in
BlobStor constructor. Implement Open/Init/Close methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
51ab6991d2
[ #220 ] blobstor: Implement method to close blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
34a259d5c0
[ #220 ] blobstor: Support zero shallow depth of blobovnicza tree
...
There is a need to support single blobovnicza in blobovnicza tree. This can
be achieved with a width of 1, and a depth of 0 or 1. With depth = 1 one
redundant directory is created, inside which there is a blobovnicza. If the
depth is zero, the blobobnivza will be in the root path. Fix negative
capacity in iterateDeepest method with zero depth.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
8bff073061
[ #220 ] blobstor: Fix hard-code in blobovnicza unit test
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
9d4fba923c
[ #199 ] Create metabase instance as storage engine component
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
8a293c76ed
[ #199 ] Ignore invalid search attributes
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
337e2f6a57
[ #199 ] Add SplitID index in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
74d44beb99
[ #199 ] Add CleanUp method to remove empty buckets from metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
7fb87aac85
[ #199 ] Support Delete operation in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
e42fa69735
[ #199 ] Make option to get object from metabase without graveyard check
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
