d8d38d3476
[ #220 ] blobstor: Add blobovnicza tree structure to BlobStor
...
Add blobovnicza instance to BlobStor structure. Create blobovnicza tree in
BlobStor constructor. Implement Open/Init/Close methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
51ab6991d2
[ #220 ] blobstor: Implement method to close blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
34a259d5c0
[ #220 ] blobstor: Support zero shallow depth of blobovnicza tree
...
There is a need to support single blobovnicza in blobovnicza tree. This can
be achieved with a width of 1, and a depth of 0 or 1. With depth = 1 one
redundant directory is created, inside which there is a blobovnicza. If the
depth is zero, the blobobnivza will be in the root path. Fix negative
capacity in iterateDeepest method with zero depth.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
8bff073061
[ #220 ] blobstor: Fix hard-code in blobovnicza unit test
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
9d4fba923c
[ #199 ] Create metabase instance as storage engine component
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
8a293c76ed
[ #199 ] Ignore invalid search attributes
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
337e2f6a57
[ #199 ] Add SplitID index in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
74d44beb99
[ #199 ] Add CleanUp method to remove empty buckets from metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
7fb87aac85
[ #199 ] Support Delete operation in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
e42fa69735
[ #199 ] Make option to get object from metabase without graveyard check
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
b45d523125
[ #199 ] Split IsSmall check in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
5bfae833fe
[ #199 ] Update blobovniczaID on existed objects in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
54cd91adff
[ #199 ] Simplify tests
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
a8d76f2ebb
[ #199 ] Implement ObjectID selection filter in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
700bd7de01
[ #199 ] Get virtual objects from metabase
...
Virtual objects are obtained differently with
relative lookup in parent bucket.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
4e7d49791b
[ #199 ] Index parent first in metabase
...
With exist check we should index parent first, because
as soon as child will be added to metabase, exist on
parent will return true even if it was not indexed yet.
Also this commit makes one db.Update instead of two for
parent and child.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f6387a9b94
[ #199 ] Make exist check on all primary buckets in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
e478c0d024
[ #199 ] Refactor metabase internal structure
...
Accoring to MetaBase-Plan-B storage engine specification.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
1c81d507fd
[ #218 ] blobstor: Inherit the root path to Blobovnicza from BlobStor
...
Place the root of blobovnicza tree in a subdirectory of BlobStor with same
permissions. Abolish WithBlobovniczaRootPath and WithBlobovniczaPersmissions
options.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4c3d231e82
[ #218 ] blobstor: Implement blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3acbd3011b
[ #218 ] blobovnicza: Сreate at least one range bucket
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1b0d2c67d1
[ #218 ] blobstor: Add blobovnicza ID value to DeleteSmallPrm
...
Blobovnicza ID parameter provides the ability to specify particular
blobovnicza to delete object from. In this case only specified blobovnicza
is processed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
a97ab242eb
[ #218 ] blobstor: Add config values of blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
35ccda13a8
[ #216 ] blobovnicza: Change ID stringer
...
Replace hex encoding with slice to string conversion.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
07e998d846
[ #216 ] blobovnicza: Implement main methods and logic
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
52b0c1f19a
[ #216 ] blobovnicza: Add configuration values
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
bce16de72a
[ #216 ] blobstor: Classify objects by size according to binary format
...
In previous implementation objects were classified by size according to
payload size. From now they are classified by the size of their binary
representation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
40b09f9266
[ #216 ] blobovnicza: Define Blobovnicza type
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
b9054e2ee0
[ #211 ] blobstor: Add a branch for processing objects by size in Put
...
Implement "big or small" property classifier (only the size of the payload
is temporarily considered). Save "big" objects in shallow dir. Save "small"
objects in shallow dir until the moment of implementation of blobovnicza.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
cfc770c3fe
[ #211 ] blobstor: Add SmallSizeLimit parameter
...
Add numeric parameter of maximum size of the "small" objects.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3028718691
[ #211 ] blobovnicza: Refactor ID implementation
...
Replace UUID implementation of ID with byte slice.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
53b114cf8b
[ #211 ] blobstor: Remove global lock
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
59f7cf9873
[ #211 ] blobstor: Implement Exists method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1ba556f5e7
[ #211 ] blobstor: Define Iterate operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3c0ef270f4
[ #211 ] blobstor: Define Exists operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
915243c69e
[ #211 ] blobstor: Define DeleteSmall operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
65da2025e5
[ #211 ] blobstor: Rename Delete to DeleteBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
b605ea3678
[ #211 ] blobstor: Refactor Put operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7afcde659d
[ #211 ] blobstor: Define GetRangeSmall operation interface
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
0b130150ae
[ #211 ] blobstor: Rename GetRange to GetRangeBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2ba4ebc8c9
[ #211 ] blobstor: Implement no-op GetSmall operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2f61d8d63c
[ #211 ] localstorage: Define blobovnicza ID
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
6813f40665
[ #211 ] blobstor: Refactor GetBig parameters and result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
fb6857a1cb
[ #211 ] blobstor: Rename Get operation to GetBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4e8cc51e7b
[ #189 ] shard: Extend Info with WeightValues
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4f16a10235
[ #189 ] localstorage: Implement DumpInfo method
...
Implement method to get the information about the local storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
8fbda2a588
[ #189 ] shard: Implement DumpInfo method
...
Implement method to get the information about the shard.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
24cf86e269
[ #189 ] metabase: Implement DumpInfo method
...
Implement method to get the information about the metabase.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
eaa7068e3c
[ #189 ] blobstor: Implement DumpInfo method
...
Implement method to get the information about the BLOB storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
dae8d3de5b
[ #189 ] shard: Fix missing return statement in SetMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
Alex Vanin