f7ca4a8dce
[ #80 ] cmd/cli: Implement storagegroup commands
...
Implement PUT/GET/LIST/DELETE commands of storage group module.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
43ee94a572
[ #80 ] storageroup: Implement function to collect structure from container
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
44466f4a29
[ #80 ] storagegroup: Implement search query generator
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
6245e58d1f
[ #80 ] services/object: Implement iterators over split-tree leaves
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
bb85ac24eb
[ #254 ] innerring: Increase extra fee for container registration
...
Some rough estimation until there will be p2p signature collection
inside the chain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
feec559e47
[ #254 ] Update neo-go to preview4 compatible version
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
c539657567
Update changelog for v0.13.2
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:34:20 +03:00
0302dc7490
Pull neofs-api-go v1.21.2
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:34:20 +03:00
51b85b0a73
[ #278 ] cli: Support request X-Headers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:03:02 +03:00
e53bf574b5
[ #279 ] container: Use new methods to work with container format
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
9680dfbdea
Update changelog for v0.13.1
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:58:38 +03:00
b56cfa4318
Pull neofs-api-go release v1.21.1
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:58:38 +03:00
bbe8f34ffd
[ #253 ] object/eacl: Process only CID/OID object headers in RANGEHASH/DELETE
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
9e27263df3
[ #253 ] object/put: Fix loss of session token
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
8e661d5c37
[ #264 ] object/get: Prevent receiving payloads through GETRANGE when GET
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
66f9532857
[ #264 ] object/get: Check parent address in linking/last child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
ec21040542
[ #264 ] transformer: Finalize parent header once
...
In previous implementation parent object header finalized twice in size
limiter + formatter. On the one hand, this added redundant action, on the
other hand, it could provoke a difference in the headers of the linking and
the last part. Change formatter to finalize parent header if it does not
container the signature. Change size limiter to reuse parent header after
last child finalization in linking child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
edef9463d7
[ #253 ] metabase: Add prefix to Graveyard and ToMoveIt bucket names
...
In previous implementation DB.Containers method could return an error about
invalid container ID string format. This could happen if some of top-level
buckets had name w/o "_" substring.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-17 12:03:18 +03:00
df3d30d1ec
Update changelog for v0.13.0
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:48:18 +03:00
49131f1bc7
[ #247 ] object/eacl: Use address from session token in request validation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
dba88c79b4
[ #247 ] object/range: Process only object_id and container_id headers
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
8654458b19
[ #247 ] object/eacl: Use object ID from session token context
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
168dcbdccd
[ #247 ] object/eacl: Process object address from request
...
In previous implementation eACL validator didn't take into account container
and object ID fields of request bodies.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
bbe700fa37
[ #250 ] service/policer: Don't shrink node list at unknown error
...
Every unknown error must not decrease shortage counter and must not
exclude faulty node from the node list, because this list will be used
later for replication.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-14 21:49:50 +03:00
7daf3727ee
[ #248 ] neofs-cli/object: Print split header values in Head command
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-14 14:46:59 +03:00
e6f2d84736
[ #246 ] blobovnivza: Fix deadlock on concurrent evict and open new blobovnicza
...
Deadlock occurs when `getActivate` function opens new blobovnicza and that
invokes evict in LRU cache of open blobovniczas. `getActivate` makes
`activeMtx.Lock()` and then cache evict makes `activeMtx.RLock()` and deadlock
happens.
Fix contains two steps:
- add separate mutex to open blobovniczas (1),
- split single Lock outside of `updateAndGet` (2).
As for the (1) `bbolt.Open()` locks when it tries to open the same file from
two threads. So separate mutex will prevent that.
As for the (2) `updateAndGet` function contains from two parts. At first it
checks if required blobovnicza is ready and it returns it. In this case we can
use the simple RLock. But then there is an option when we should open new
blobovnicza and update map of active blobovniczas.
In this case we call `openBlobovnicza` without activeMtx lock. Cache evict
happens there and it won't cause deadlock.
Then we lock activeMtx to update the map of active blobovniczas. Concurrency can
happen there. However `openBlobovnicza` will not open the same blobovnicza twice,
so we can make one more check if opened blobovnicza was activated while thread was
locked in activeMtx. If so, then return active blobovnicza, else finish activation.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-14 13:37:45 +03:00
91bea44a1a
[ #245 ] blobovnicza: Fix initializing dimensional buckets
...
In previous implementation Blobovnicza could incorrectly initialize
dimensional buckets: if SmallSizeLimit = 2 ^ X + Y && Y < 2 ^ X, then
largest dimensional bucket was [2 ^ (X - 1) : 2 ^ X]. This was caused by an
incorrect condition for stopping the iterator along the dimensional
boundaries.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-14 12:48:59 +03:00
6814140a19
[ #240 ] cmd/cli: Support raw flag in object GET/HEAD/RANGE commands
...
Support boolean raw flag. Print split information upon receipt.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-14 12:05:13 +03:00
fb50362dcc
[ #243 ] services/object: Share common parameters across services
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
a01262d8bd
[ #243 ] service/object: Fix object chain assembly for tombstone body
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
9265e31e65
[ #243 ] services/object: Fix lost tokens when generating requests
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
09916f21b5
[ #243 ] cmd/cli: Write tombstone address in Object.Delete cmd
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
fe3906c295
[ #243 ] object/delete: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
510e9ff2ec
[ #243 ] core/object: Replace test content of tombstone with an API structure
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
4f5d5c7e45
Write system path to BoltDB on blobovnicza closing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
627012e4fe
[ #241 ] object/search: Set container ID parameter of storage engine call
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
611a29f682
[ #241 ] object/search: Refactor service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
3bfb18386b
object/get: Replace code with function call
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
63bc3aab82
[ #242 ] engine: Do not iterate over shards if cid is not set
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
9fb7190358
[ #242 ] service/object: Support ContainerID field in local search
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
511b5f782a
[ #242 ] engine: Support ContainerID argument in shard selects
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
f585772746
[ #242 ] shard: Support ContainerID argument in metabase selects
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
b97f818323
[ #242 ] metabase: Make ContainerID a mandatory parameter for Select
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
0e1f05ff45
[ #239 ] object/head: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
9dd83bdf0d
[ #235 ] blobstor: Return object.ErrRangeOutOfBounds from shallow dir
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
41b9fa5b45
[ #235 ] engine: Return OutOfBounds error
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
173d34a8a2
[ #235 ] object/get: Set common operation parameters
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
7e56427534
[ #235 ] object/getrangehash: Implement new service processing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
0fc5ea674c
[ #235 ] object/getrange: Fix incorrect processing of range from last child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
397d912e19
[ #235 ] object/head: Support raw flag in service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
Alex Vanin