Leonard Lyubich
35ccda13a8
[ #216 ] blobovnicza: Change ID stringer
...
Replace hex encoding with slice to string conversion.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
07e998d846
[ #216 ] blobovnicza: Implement main methods and logic
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
52b0c1f19a
[ #216 ] blobovnicza: Add configuration values
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
bce16de72a
[ #216 ] blobstor: Classify objects by size according to binary format
...
In previous implementation objects were classified by size according to
payload size. From now they are classified by the size of their binary
representation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
40b09f9266
[ #216 ] blobovnicza: Define Blobovnicza type
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
a1164b2a5c
[ #211 ] cmd/neofs-node: Add SmallSizeLimit config value of BlobStor
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
b9054e2ee0
[ #211 ] blobstor: Add a branch for processing objects by size in Put
...
Implement "big or small" property classifier (only the size of the payload
is temporarily considered). Save "big" objects in shallow dir. Save "small"
objects in shallow dir until the moment of implementation of blobovnicza.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
cfc770c3fe
[ #211 ] blobstor: Add SmallSizeLimit parameter
...
Add numeric parameter of maximum size of the "small" objects.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
3028718691
[ #211 ] blobovnicza: Refactor ID implementation
...
Replace UUID implementation of ID with byte slice.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
53b114cf8b
[ #211 ] blobstor: Remove global lock
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
59f7cf9873
[ #211 ] blobstor: Implement Exists method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1ba556f5e7
[ #211 ] blobstor: Define Iterate operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
3c0ef270f4
[ #211 ] blobstor: Define Exists operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
915243c69e
[ #211 ] blobstor: Define DeleteSmall operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
65da2025e5
[ #211 ] blobstor: Rename Delete to DeleteBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
b605ea3678
[ #211 ] blobstor: Refactor Put operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
7afcde659d
[ #211 ] blobstor: Define GetRangeSmall operation interface
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
0b130150ae
[ #211 ] blobstor: Rename GetRange to GetRangeBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2ba4ebc8c9
[ #211 ] blobstor: Implement no-op GetSmall operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2f61d8d63c
[ #211 ] localstorage: Define blobovnicza ID
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
6813f40665
[ #211 ] blobstor: Refactor GetBig parameters and result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
fb6857a1cb
[ #211 ] blobstor: Rename Get operation to GetBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
4e8cc51e7b
[ #189 ] shard: Extend Info with WeightValues
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
4f16a10235
[ #189 ] localstorage: Implement DumpInfo method
...
Implement method to get the information about the local storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
8fbda2a588
[ #189 ] shard: Implement DumpInfo method
...
Implement method to get the information about the shard.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
24cf86e269
[ #189 ] metabase: Implement DumpInfo method
...
Implement method to get the information about the metabase.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
eaa7068e3c
[ #189 ] blobstor: Implement DumpInfo method
...
Implement method to get the information about the BLOB storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
dae8d3de5b
[ #189 ] shard: Fix missing return statement in SetMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
cba87a1300
[ #188 ] localstorage: Implement SetShardMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
3fa3661cad
[ #188 ] shard: Implement SetMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2fb379b7dd
[ #188 ] shard: Define the enumeration of shard modes
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
0b515837ca
[ #186 ] localstorage: Fix address forwarding in Delete method
...
In previous implementation storage engine not forwarded object address to
shards.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
4ca1035aa4
[ #186 ] cmd/neofs-node: Integrate new storage engine into application
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
953387a1e5
[ #186 ] object/acl: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
046206f670
[ #186 ] object/head: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2be8f154a0
[ #186 ] object/put: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
14442a0801
[ #186 ] object/range: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
9b32f131c0
[ #186 ] object/rangehash: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1bce2a4e55
[ #186 ] object/search: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
aa9eb2eaf2
[ #186 ] policer: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1b867e53d7
[ #186 ] replicator: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
9a4cd01eb8
[ #186 ] localstorage: Implement utility functions over StorageEngine
...
Implement Put/Get/GetRange/Select/SelectAll functions over storage
engine. These functions are going to be used by Object service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
70a65c5aff
[ #176 ] blobstor: Use 1 byte names of sub-directories in FS tree
...
Using 2 bytes length for directories in fs tree lead to large growth
in the number of directories.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
2d917a024f
[ #176 ] blobstor: Fix missing mutex in constructor
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
b1718a60e4
[ #176 ] shard: Fix BlobStor access when data range is zero
...
In previous implementation Shard accessed the BlobStor to get the
object header. However, the shard must take headers from the metabase.
From now zero length of the requested payload range seens as object
header request. In this case shard calls metabase to get the header.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
796bba3793
[ #176 ] shard: Finalize the implementation of Exists method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
4518357ad7
[ #176 ] localstore: Fix linter's remarks
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
f194c840d7
[ #176 ] blobstor: Handle error of zstd (de)compressor creation
...
In previous implementation WithCompressObjects returned Option
than could panic if zstd (de)compressor creation failed with error.
From now errors with (de)compressor creation result in an option
without using data compression. In this case, the error is written
to the log passed to the option constructor.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
6f8c45d61b
[ #176 ] localstore: Change multiple access sync
...
In previous implementation each operation on local storage
locked engine mutex. This was done under the assumption that
the weights of the shards change as a result of write operations.
With the transition to static weights of shards, it is no longer
necessary to lock the global mutex during the execution of operations.
However, since the set of engine shards is dynamic, there is still a
need to control multiple access to this set. The same mutex is used
for synchronization.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
df558cbe6b
[ #176 ] shard: Get rid of using the global RW lock
...
In previous implementation each shard operation locked
RW shard mutex. With this approach RW operations were executed
one-by-one and blocked the execution of RO operations.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
