bb85ac24eb
[ #254 ] innerring: Increase extra fee for container registration
...
Some rough estimation until there will be p2p signature collection
inside the chain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
feec559e47
[ #254 ] Update neo-go to preview4 compatible version
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
e53bf574b5
[ #279 ] container: Use new methods to work with container format
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
bbe8f34ffd
[ #253 ] object/eacl: Process only CID/OID object headers in RANGEHASH/DELETE
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
9e27263df3
[ #253 ] object/put: Fix loss of session token
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
8e661d5c37
[ #264 ] object/get: Prevent receiving payloads through GETRANGE when GET
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
66f9532857
[ #264 ] object/get: Check parent address in linking/last child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
ec21040542
[ #264 ] transformer: Finalize parent header once
...
In previous implementation parent object header finalized twice in size
limiter + formatter. On the one hand, this added redundant action, on the
other hand, it could provoke a difference in the headers of the linking and
the last part. Change formatter to finalize parent header if it does not
container the signature. Change size limiter to reuse parent header after
last child finalization in linking child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
edef9463d7
[ #253 ] metabase: Add prefix to Graveyard and ToMoveIt bucket names
...
In previous implementation DB.Containers method could return an error about
invalid container ID string format. This could happen if some of top-level
buckets had name w/o "_" substring.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-17 12:03:18 +03:00
49131f1bc7
[ #247 ] object/eacl: Use address from session token in request validation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
dba88c79b4
[ #247 ] object/range: Process only object_id and container_id headers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
8654458b19
[ #247 ] object/eacl: Use object ID from session token context
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
168dcbdccd
[ #247 ] object/eacl: Process object address from request
...
In previous implementation eACL validator didn't take into account container
and object ID fields of request bodies.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
bbe700fa37
[ #250 ] service/policer: Don't shrink node list at unknown error
...
Every unknown error must not decrease shortage counter and must not
exclude faulty node from the node list, because this list will be used
later for replication.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-14 21:49:50 +03:00
e6f2d84736
[ #246 ] blobovnivza: Fix deadlock on concurrent evict and open new blobovnicza
...
Deadlock occurs when `getActivate` function opens new blobovnicza and that
invokes evict in LRU cache of open blobovniczas. `getActivate` makes
`activeMtx.Lock()` and then cache evict makes `activeMtx.RLock()` and deadlock
happens.
Fix contains two steps:
- add separate mutex to open blobovniczas (1),
- split single Lock outside of `updateAndGet` (2).
As for the (1) `bbolt.Open()` locks when it tries to open the same file from
two threads. So separate mutex will prevent that.
As for the (2) `updateAndGet` function contains from two parts. At first it
checks if required blobovnicza is ready and it returns it. In this case we can
use the simple RLock. But then there is an option when we should open new
blobovnicza and update map of active blobovniczas.
In this case we call `openBlobovnicza` without activeMtx lock. Cache evict
happens there and it won't cause deadlock.
Then we lock activeMtx to update the map of active blobovniczas. Concurrency can
happen there. However `openBlobovnicza` will not open the same blobovnicza twice,
so we can make one more check if opened blobovnicza was activated while thread was
locked in activeMtx. If so, then return active blobovnicza, else finish activation.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-14 13:37:45 +03:00
91bea44a1a
[ #245 ] blobovnicza: Fix initializing dimensional buckets
...
In previous implementation Blobovnicza could incorrectly initialize
dimensional buckets: if SmallSizeLimit = 2 ^ X + Y && Y < 2 ^ X, then
largest dimensional bucket was [2 ^ (X - 1) : 2 ^ X]. This was caused by an
incorrect condition for stopping the iterator along the dimensional
boundaries.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-14 12:48:59 +03:00
fb50362dcc
[ #243 ] services/object: Share common parameters across services
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
a01262d8bd
[ #243 ] service/object: Fix object chain assembly for tombstone body
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
9265e31e65
[ #243 ] services/object: Fix lost tokens when generating requests
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
fe3906c295
[ #243 ] object/delete: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
510e9ff2ec
[ #243 ] core/object: Replace test content of tombstone with an API structure
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4f5d5c7e45
Write system path to BoltDB on blobovnicza closing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
627012e4fe
[ #241 ] object/search: Set container ID parameter of storage engine call
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
611a29f682
[ #241 ] object/search: Refactor service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3bfb18386b
object/get: Replace code with function call
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
63bc3aab82
[ #242 ] engine: Do not iterate over shards if cid is not set
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
9fb7190358
[ #242 ] service/object: Support ContainerID field in local search
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
511b5f782a
[ #242 ] engine: Support ContainerID argument in shard selects
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f585772746
[ #242 ] shard: Support ContainerID argument in metabase selects
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
b97f818323
[ #242 ] metabase: Make ContainerID a mandatory parameter for Select
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
0e1f05ff45
[ #239 ] object/head: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
9dd83bdf0d
[ #235 ] blobstor: Return object.ErrRangeOutOfBounds from shallow dir
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
41b9fa5b45
[ #235 ] engine: Return OutOfBounds error
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
173d34a8a2
[ #235 ] object/get: Set common operation parameters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7e56427534
[ #235 ] object/getrangehash: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
0fc5ea674c
[ #235 ] object/getrange: Fix incorrect processing of range from last child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
397d912e19
[ #235 ] object/head: Support raw flag in service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1d23483828
[ #235 ] services/object: Implement new GetRange algorithm
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
91d8e0a4de
[ #238 ] engine: Support raw flag in Head method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
da42ee3a26
[ #238 ] engine: Stop shard iteration at split info error in Head
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
69dee6ec6d
[ #238 ] shard: Support raw flag in Head method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
bf7a390304
[ #238 ] metabase: Support raw flag in Get method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
c267a08886
[ #236 ] engine: Stop shard iteration at split info error in GetRange
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
590745204c
[ #237 ] metabase: Structure parameters and results of all operations
...
All parameters and resulting values of all metabase operations are
structured in new types. The most popular scenarios for using operations are
moved to auxiliary functions.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
a875d80491
Remove outdated code of metabase and localstore
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
869d9e571c
[ #233 ] object/head: Remove getting the right child header
...
With the update of the local storage engine, the headers of virtual objects
are directly given. In this regard, the step with obtaining the the right
child header is removed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
f24daa10ff
[ #233 ] services/object: Implement new Get algorithm
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
26f03c6301
[ #232 ] Fix SplitInfo update in existed virtual object
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
e0350efe00
[ #231 ] services/policer: Use engine.List method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
ee1e3931a5
[ #231 ] engine: Add List method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
a7087faa1a
[ #231 ] shard: Add List method to return all available objects
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
a1cb48f800
[ #231 ] metabase: Add container lister
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
3a37eda410
[ #230 ] engine: Stop on first SplitInfoError in Get method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
a488d202e1
[ #230 ] engine: Rename `WithAddress` to `WithAddresses` for delete operations
...
Delete works with multiple addresses at once so it is convenient to
show it in parameter setter function.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
7c97bd1360
[ #230 ] shard: Add test for getting SplitInfo
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
bf9e938a3b
[ #230 ] metabase: Return SplitInfoError in Exist method
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
e2de95e3f6
[ #230 ] metabase: Index split info structures
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
eaae5a5dd7
[ #229 ] blobovnicza: Store objects in a binary format
...
In previous implementation Blobovnicza's stored objects in protocol format
which did not allow working with externally compressed objects. To achieve
this goal, operations Get and Put no longer work with the structure of the
object, but only with abstract binary data. Operation GetRange has become
incorrect in its original purpose to receive the payload range. In this
regard, BlobStor receives the payload range of the object through Get
operation. In the future either Blobovnicza will learn to compress objects
by itself, or the GetRange operation will be eliminated.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3d77fdb347
[ #222 ] Set write cache specific config on application level
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
af2719689d
[ #222 ] Do not compress data in blobovniczas
...
Blobovnicza returns object, so we can't put compressed
data there. Compressed data won't be deserialized correctly.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
6bec50963b
[ #222 ] Add tests for shards
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
ce92df3543
[ #222 ] Return nil in `metabase.IsSmall` if object is big
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
862f9a3f1a
[ #222 ] Put small objects in blobovnicza in blobStor
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
351e4b4592
[ #222 ] Support Inhume and Delete in object service
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
21708d5408
[ #222 ] Add Exists method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f470022594
[ #222 ] Update Select method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
25486194b9
[ #222 ] Update Head method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
67d4b0db12
[ #222 ] Move ErrAlreadyRemoved to core package
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
6a3e3e5642
[ #222 ] Update Range method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
c9b1c6603e
[ #222 ] Update Get method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f3e96f8c19
[ #222 ] Update Delete method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
aa8dd18b07
[ #222 ] Update Put method in storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f45b7a048d
[ #222 ] Rename `WithTreeRoot..` to `WithRoot..` in blobstor opts
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
d4e48b28b1
[ #222 ] Add Inhume method to storage engine
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
92c95150cd
[ #222 ] Use write cache options in storage app
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
69ba295077
[ #222 ] Add ToMoveIt method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
ccd8760fbc
[ #222 ] Update Exists method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
97a2e26b71
[ #222 ] Add Head method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
244e244c0e
[ #222 ] Add GetRange method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
73505fe582
[ #222 ] Update Get method in shard
...
Now `Get`, `Head` and `GetRange` will be
separate methods on shard.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
31371771e8
[ #222 ] Update Delete method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
5fc54386b0
[ #222 ] Update Put method in shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
72f8ca1afd
[ #222 ] Add Inhume method for shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
1b76458684
[ #222 ] Add writeCache instance to shard
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
cbdcfb9159
[ #220 ] shard: Enable write cache through option
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
47d2239332
[ #220 ] localstorage: Replace basic errors to core library
...
Replace ErrNotFound and ErrRangeOutOfBounds to core/object package in order
to share them across the libraries.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7512a5ba18
[ #220 ] blobstor: Implement "small" operations
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7f20e6f9db
[ #220 ] blobstor: Implement GetRange on blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
ef095d4871
[ #220 ] blobovnicza: Implement GetRange operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
e758c246b7
[ #220 ] localstorage: Implement Open/Init/Close methods
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2963473c08
[ #220 ] localstorage: Implement hrw.Hasher over Shard structure
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
88c1584e6a
[ #220 ] shard: Implement Open/Init/Close methods
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
d30454a572
[ #220 ] shard: Use new metabase implementation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
d8d38d3476
[ #220 ] blobstor: Add blobovnicza tree structure to BlobStor
...
Add blobovnicza instance to BlobStor structure. Create blobovnicza tree in
BlobStor constructor. Implement Open/Init/Close methods.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
51ab6991d2
[ #220 ] blobstor: Implement method to close blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
34a259d5c0
[ #220 ] blobstor: Support zero shallow depth of blobovnicza tree
...
There is a need to support single blobovnicza in blobovnicza tree. This can
be achieved with a width of 1, and a depth of 0 or 1. With depth = 1 one
redundant directory is created, inside which there is a blobovnicza. If the
depth is zero, the blobobnivza will be in the root path. Fix negative
capacity in iterateDeepest method with zero depth.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
8bff073061
[ #220 ] blobstor: Fix hard-code in blobovnicza unit test
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
9d4fba923c
[ #199 ] Create metabase instance as storage engine component
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
8a293c76ed
[ #199 ] Ignore invalid search attributes
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
337e2f6a57
[ #199 ] Add SplitID index in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
74d44beb99
[ #199 ] Add CleanUp method to remove empty buckets from metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
7fb87aac85
[ #199 ] Support Delete operation in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
e42fa69735
[ #199 ] Make option to get object from metabase without graveyard check
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
b45d523125
[ #199 ] Split IsSmall check in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
5bfae833fe
[ #199 ] Update blobovniczaID on existed objects in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
54cd91adff
[ #199 ] Simplify tests
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
a8d76f2ebb
[ #199 ] Implement ObjectID selection filter in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
700bd7de01
[ #199 ] Get virtual objects from metabase
...
Virtual objects are obtained differently with
relative lookup in parent bucket.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
4e7d49791b
[ #199 ] Index parent first in metabase
...
With exist check we should index parent first, because
as soon as child will be added to metabase, exist on
parent will return true even if it was not indexed yet.
Also this commit makes one db.Update instead of two for
parent and child.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f6387a9b94
[ #199 ] Make exist check on all primary buckets in metabase
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
e478c0d024
[ #199 ] Refactor metabase internal structure
...
Accoring to MetaBase-Plan-B storage engine specification.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
1c81d507fd
[ #218 ] blobstor: Inherit the root path to Blobovnicza from BlobStor
...
Place the root of blobovnicza tree in a subdirectory of BlobStor with same
permissions. Abolish WithBlobovniczaRootPath and WithBlobovniczaPersmissions
options.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4c3d231e82
[ #218 ] blobstor: Implement blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3acbd3011b
[ #218 ] blobovnicza: Сreate at least one range bucket
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1b0d2c67d1
[ #218 ] blobstor: Add blobovnicza ID value to DeleteSmallPrm
...
Blobovnicza ID parameter provides the ability to specify particular
blobovnicza to delete object from. In this case only specified blobovnicza
is processed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
a97ab242eb
[ #218 ] blobstor: Add config values of blobovnicza tree
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
35ccda13a8
[ #216 ] blobovnicza: Change ID stringer
...
Replace hex encoding with slice to string conversion.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
07e998d846
[ #216 ] blobovnicza: Implement main methods and logic
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
52b0c1f19a
[ #216 ] blobovnicza: Add configuration values
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
bce16de72a
[ #216 ] blobstor: Classify objects by size according to binary format
...
In previous implementation objects were classified by size according to
payload size. From now they are classified by the size of their binary
representation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
40b09f9266
[ #216 ] blobovnicza: Define Blobovnicza type
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
b9054e2ee0
[ #211 ] blobstor: Add a branch for processing objects by size in Put
...
Implement "big or small" property classifier (only the size of the payload
is temporarily considered). Save "big" objects in shallow dir. Save "small"
objects in shallow dir until the moment of implementation of blobovnicza.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
cfc770c3fe
[ #211 ] blobstor: Add SmallSizeLimit parameter
...
Add numeric parameter of maximum size of the "small" objects.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3028718691
[ #211 ] blobovnicza: Refactor ID implementation
...
Replace UUID implementation of ID with byte slice.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
53b114cf8b
[ #211 ] blobstor: Remove global lock
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
59f7cf9873
[ #211 ] blobstor: Implement Exists method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
1ba556f5e7
[ #211 ] blobstor: Define Iterate operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3c0ef270f4
[ #211 ] blobstor: Define Exists operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
915243c69e
[ #211 ] blobstor: Define DeleteSmall operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
65da2025e5
[ #211 ] blobstor: Rename Delete to DeleteBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
b605ea3678
[ #211 ] blobstor: Refactor Put operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7afcde659d
[ #211 ] blobstor: Define GetRangeSmall operation interface
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
0b130150ae
[ #211 ] blobstor: Rename GetRange to GetRangeBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2ba4ebc8c9
[ #211 ] blobstor: Implement no-op GetSmall operation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2f61d8d63c
[ #211 ] localstorage: Define blobovnicza ID
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
6813f40665
[ #211 ] blobstor: Refactor GetBig parameters and result
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
fb6857a1cb
[ #211 ] blobstor: Rename Get operation to GetBig
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4e8cc51e7b
[ #189 ] shard: Extend Info with WeightValues
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4f16a10235
[ #189 ] localstorage: Implement DumpInfo method
...
Implement method to get the information about the local storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
8fbda2a588
[ #189 ] shard: Implement DumpInfo method
...
Implement method to get the information about the shard.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
24cf86e269
[ #189 ] metabase: Implement DumpInfo method
...
Implement method to get the information about the metabase.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
eaa7068e3c
[ #189 ] blobstor: Implement DumpInfo method
...
Implement method to get the information about the BLOB storage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
dae8d3de5b
[ #189 ] shard: Fix missing return statement in SetMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
cba87a1300
[ #188 ] localstorage: Implement SetShardMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3fa3661cad
[ #188 ] shard: Implement SetMode method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2fb379b7dd
[ #188 ] shard: Define the enumeration of shard modes
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
0b515837ca
[ #186 ] localstorage: Fix address forwarding in Delete method
...
In previous implementation storage engine not forwarded object address to
shards.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
953387a1e5
[ #186 ] object/acl: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
046206f670
[ #186 ] object/head: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
2be8f154a0
[ #186 ] object/put: Use new storage engine for work
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
Leonard Lyubich