Commit Graph

2583 Commits (f6d121a4e40a936273e8cd08bc8384abffcc4f2b)

Author SHA1 Message Date
Pavel Karpy 90a8c52bdb [#1255] object: Add persistent storage usage
Use persistent storage usage in the node if it was configured so.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Pavel Karpy 9cda3121ab [#1255] node/config: Add persistent storage
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Pavel Karpy 016eaa25f3 [#1255] node/session: Add encryption tests
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Pavel Karpy 01ed366e99 [#1255] node/session: Add encryption
Add `WithEncryption` option that passes ECDSA key to the persistent session
storage. It uses 32 bytes from marshalled ECDSA key in ASN.1 DER from in
AES-256 algorithm encryption in Galois/Counter Mode.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Pavel Karpy a884ad56d9 [#1255] node/session: Add persistent tests
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Pavel Karpy 455b9fb325 [#1255] node/session: Add persistent session storage
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Pavel Karpy 929c9851a6 [#1255] node/session: Create separate dir for in-memory storage
Move in-memory session storage to the separate directory of `storage`. It is
done for future support of different kind of session storages.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-29 09:35:10 +03:00
Evgenii Stratonikov 2a69aaf976 [#1157] network/cache: Optimize `client` fetch from `multiClient`
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 18:11:20 +03:00
Evgenii Stratonikov a4261243fc [#1157] network/cache: Cache multiclients based on public key only
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 18:11:20 +03:00
Evgenii Stratonikov de5a2f6574 [#1262] metabase: Remove list index in place `Delete`
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
Evgenii Stratonikov 43867a3093 [#1262] metabase: Do not allocate intermediate slices for indices
```
name              old alloc/op   new alloc/op   delta
Put/parallel-8       123kB ± 4%     119kB ± 3%  -2.72%  (p=0.006 n=10+9)
Put/sequential-8     170kB ± 1%     168kB ± 1%  -1.42%  (p=0.000 n=10+10)

name              old allocs/op  new allocs/op  delta
Put/parallel-8         473 ± 1%       469 ± 0%  -0.87%  (p=0.000 n=10+10)
Put/sequential-8       792 ± 0%       787 ± 0%  -0.58%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
Evgenii Stratonikov 0e9b6be3fd [#1262] metabase: Remove intermediate allocations in `decodeList`
```
name              old time/op    new time/op    delta
Put/parallel-8      1.57ms ±11%    1.51ms ± 3%   -4.06%  (p=0.043 n=9+10)
Put/sequential-8    5.16ms ± 2%    5.16ms ± 3%     ~     (p=1.000 n=9+10)

name              old alloc/op   new alloc/op   delta
Put/parallel-8       126kB ± 4%     123kB ± 4%   -2.54%  (p=0.016 n=8+10)
Put/sequential-8     171kB ± 1%     170kB ± 1%     ~     (p=0.182 n=9+10)

name              old allocs/op  new allocs/op  delta
Put/parallel-8         565 ± 2%       473 ± 1%  -16.18%  (p=0.000 n=9+10)
Put/sequential-8       819 ± 1%       792 ± 0%   -3.34%  (p=0.000 n=9+10)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
Evgenii Stratonikov d45df614fb [#1262] metabase: Optimize `decodeList`
Prevent additional allocation during `append` in `Put`.

```
name              old alloc/op   new alloc/op   delta
Put/parallel-8       131kB ± 1%     126kB ± 4%  -3.87%  (p=0.005 n=8+8)
Put/sequential-8     172kB ± 1%     171kB ± 1%  -0.73%  (p=0.028 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
Evgenii Stratonikov 456e1584d6 [#1262] metabase: Add benchmarks for `Put`
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
Evgenii Stratonikov f1223b46df [#1262] blobovnicza: Make helper in `Put` function idempotent
`Batch` can execute the function multiple times leading to multiple
increases of a size approximation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
Alex Vanin 35ad6f188e Backport release v0.27.6 changelog
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-28 15:16:54 +03:00
Evgenii Stratonikov 571ae843ad [#1198] neofs-cli: Use `io.Copy` instead of `io.CopyBuffer`
The buffer size value is somewhat arbitrary and making in configurable
doesn't make much sense, given that we can't really restrict total resource
consumption in other places. `Copy` uses 32 KiB buffer by default, which
is not big. This approach is also more flexible as we can get rid of
buffer completely by implementing `ReaderFrom`, `WriterTo` interfaces.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-25 18:21:48 +03:00
Evgenii Stratonikov cbe07120da [#1261] neofs-cli: Allow to use relative epoch for bearer token
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-25 17:58:53 +03:00
Evgenii Stratonikov f2c1bc4bfb [#1261] neofs-cli: Allow to create eACL with empty container ID
Empty CID can be used in bearer token eACL.
See https://github.com/nspcc-dev/neofs-api/issues/207 .

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-25 17:58:53 +03:00
Evgenii Stratonikov 9b2523a408 [#1261] neofs-cli: Allow to create bearer tokens
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-25 17:58:53 +03:00
Evgenii Stratonikov ae8e38cace [#1261] neofs-cli: Fix help message for `acl extended create`
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-25 17:58:53 +03:00
Alex Vanin 683439970a [#1270] neofs-node: Add timeout for grpc GracefulStop()
GracefulStop() may be blocked until all server-side streams
are finished. There is no control over such streams yet, so
application may be frozen in shutdown stage.

Naive solution is to add timeout for GracefulStop(). At this
point healthy connection will be finished and unhealthy
connections will be terminated by Stop().

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-25 17:39:24 +03:00
Evgenii Stratonikov ad92493b86 [#1268] blobstor: Cleanup zstd encoders/decoders
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-25 14:05:06 +03:00
Evgenii Stratonikov 4253931699 Release candidate v0.28.0-rc.2
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-24 17:51:59 +03:00
Alex Vanin 7e06d0aa69 [#1253] ir: Call UpdateStateIR method to remove dead storage node
Alphabet nodes in notary enabled environment cannot call `UpdateState`
method to remove unwanted storage nodes from the network map,
because this method checks witness of the storage node.

To force storage node state update, alphabet nodes should invoke
new method `UpdateStateIR` which is similar to `AddPeerIR`.

State update initiated by the storage node itself is processed
the same way as before -- alphabet nods resign such transaction.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-24 11:52:19 +03:00
Alex Vanin e81081e0e0 [#1253] ir: Update AddPeer method name for notary enabled env
`Register` was renamed to `AddPeerIR` for consistency with
`UpdateState` changes in
https://github.com/nspcc-dev/neofs-contract/pull/227

This is protocol breaking change for notary enabled environment.
Luckily, there is no notary enabled environments anywhere except
of neofs-dev-env, so we can do such thing. We should avoid such
changes in the future, though.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-24 11:52:19 +03:00
Evgenii Stratonikov 414ba6e0a2 [#1244] nats: Split client creation into 2 stages
Create and connect to an endpoint using separate functions.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-24 11:51:49 +03:00
Evgenii Stratonikov 2b0460c532 [#1233] neofs-cli: Fix split info marshaling
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-23 14:52:49 +03:00
Evgenii Stratonikov e2062013cf network: remove unused constants
Fix linter complaints. These constants are unused after
nspcc-dev/neofs-node#1232.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-22 11:55:19 +03:00
Alex Vanin c8b585b991 [#1259] neofs-cli: Use more `cmd.PrintErr*()`
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-21 19:20:19 +03:00
Alex Vanin 44138adacf [#1259] neofs-cli: Return non-zero exit code in `acl extended create` command failures
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-21 19:20:19 +03:00
Evgenii Stratonikov 32badab11a [#1252] neofs-cli: Print details for AccessDenied errors
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-21 19:20:01 +03:00
Evgenii Stratonikov 2848001dfb [#1246] object/acl: Return more concise description for eACL errors
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-21 19:20:01 +03:00
Evgenii Stratonikov 800d01e28c [#1233] neofs-cli: Do not print info if output format is strict
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-21 14:52:05 +03:00
Evgenii Stratonikov 5eef0f46c5 [#1233] neofs-cli: Respect format flags for `SplitInfo` output
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-21 14:52:05 +03:00
Alex Vanin 5c5279688b Release candidate v0.28.0-rc.1
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-18 10:17:08 +03:00
Alex Vanin 7c5cdd2144 Update neofs-sdk-go to latest master
Fixes missing message text in internal errors

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-18 10:17:08 +03:00
Leonard Lyubich 318639e5bf [#1247] *: Clarify docs about returned errors from `apistatus` package
`apistatus` package provides types which implement build-in `error`
interface. Add `error of type` pattern when documenting these errors in
order to clarify how these errors should be handled (e.g. `errors.Is` is
not good).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich 459bdcf04b [#1247] object/acl: Return `ObjectAccessDenied` status error
Return `apistatus.ObjectAccessDenied` error on access violation from ACL
service. Write reason in format of the errors from the previous
implementation. These errors are returned by storage node's server as
NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich 967650f2ed [#1247] container: Return `ContainerNotFound` status error
Replace `core/container.ErrNotFound` error returned by `Source.Get`
interface method with `apistatus.ContainerNotFound` status error. This
error is returned by storage node's server as NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich 70ffdf3478 [#1247] object: Return `NOT_FOUND` and `ALREADY_REMOVED` statuses
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich f32c9670ad [#1247] object: Return status errors from `util.KeyStorage`
Return `SessionTokenExpired`/`SessionTokenNotFound` error from
`apistatus` package if private session token is expired/missing. These
errors are returned by storage node's server as NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich a4ee59977e [#1247] session: Remove unused `storage.ErrNotFound` error
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Evgenii Stratonikov 737d40f2d9 [#1152] network: Reuse URI parsing function from api-go
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-17 16:33:08 +03:00
Pavel Karpy 52e742bac1 [#1243] node: Add "hot" notifications
Wrap engine with notifications writer (if configured so) to allow sending
notifications right after the object is saved in the local storage.
"Hot" notifications are sent for objects with the following tick epoch
values:
1. 0;
2. *current epoch*.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-17 14:27:52 +03:00
Pavel Karpy 5dba64fcc5 [#1243] node/cfg: Add notification config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-17 14:27:52 +03:00
Pavel Karpy 16543a1904 [#1243] node/object: Hide engine behind interface
Do not use engine as explicit dependency in the put service. This allows
wrapping storage engine.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-17 14:27:52 +03:00
Pavel Karpy a4a29f3442 [#1243] node: Rename notifications init func
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-17 14:27:52 +03:00
Pavel Karpy d02720c910 [#1243] nats: Fix comments
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-17 14:27:52 +03:00
Pavel Karpy 7ebeb4c89b [#1243] node/object: Fix comment
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-17 14:27:52 +03:00