acid-ant/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
2609 commits 110 branches 26 tags 19 MiB
Commit graph

2609 commits

This branch
This branch
All branches
Author SHA1 Message Date
Leonard Lyubich
629a4f79cd [#1293] metabase: Fix out-of-range panic in freePotentialLocks 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-04-05 15:38:51 +03:00
Alex Vanin
e62e02815b [#1291] listener: Stop only once 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-05 14:46:40 +03:00
Alex Vanin
77f4a5844b [#1291] innerring: Don't stop the same listener twice 
When IR node configured without main chain, both
`morphListener` and `mainnetListener` are pointing
into single listener component. We should not call
`Stop()` twice, because it may trigger channel
closing in neo-go or other components and it
can throw panic.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-05 14:46:40 +03:00
Evgenii Stratonikov
d311585de6 [#1289] neofs-adm: Allow to change parameters of the Policy contract 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-05 13:28:42 +03:00
Evgenii Stratonikov
a5cf38dcbf [#1288] neofs-adm: Allow to store garbage in alphabet wallet dir 
Check the full file name instead of just prefix. Also, fix a bug where
a single missing wallet could lead to an incorrect size calculation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-05 13:11:50 +03:00
Evgenii Stratonikov
923db59149 [#1213] morph/client: Remember last succesfully used client 
Prevent attemts to connect to the failing endpoint multiple times.
This speeds up all invocations a bit and removes excessive error logs.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-01 17:07:12 +03:00
Evgenii Stratonikov
9bbb136e4a [#1213] morph/client: Protect cached values with a mutex 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-01 17:07:12 +03:00
Evgenii Stratonikov
a8d2001b35 [#1213] morph/client: Use a separate cache for every client 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-01 17:07:12 +03:00
Evgenii Stratonikov
61f60b8461 [#1213] morph/client: Unlock mutex to perform network requests 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-01 17:07:12 +03:00
Evgenii Stratonikov
48eb87d32c [#1149] writecache: Prevent corruption in Head 
Also, remove optimization comments:
1. Having to maintain an execute the same logic for headers as for
   objects is quite inefficient, as it increases memory footprint.
2. Unmarshaling object is a cheap operation if data slice is in memory.
3. For unmarshaling header-only, I think we need SDK support.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:40:25 +03:00
Evgenii Stratonikov
ff1912aa2a services/acl: check session token expiration epoch 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:37:29 +03:00
Evgenii Stratonikov
68903c9fd9 [#1143] shard: Support degraded mode in Get and GetRange 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Evgenii Stratonikov
f058cead8f [#1143] shard: Handle some errors in degraded mode 
If metabase is corrupted for some reason, failback to
checking blobstor directly.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Evgenii Stratonikov
08e7914729 [#1143] blobstor: Implement existsSmall check 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Evgenii Stratonikov
aa0cc1f824 [#1143] blobovnicza: Copy object data in Get 
Data returned from `*bbolt.Bucket.Get()` is only valid for the lifetime
of the transaction.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Evgenii Stratonikov
6472a170eb [#1143] shard: Introduce explicit Degraded mode 
`Degraded` mode is set automatically after error counter is over the
threshold. `ReadOnly` mode can still be set by an administrator.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Alex Vanin
9eb70c18c3 Backport release v0.27.7 changelog 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-30 15:40:13 +03:00
Alex Vanin
e4a8ed589b [#1278] neofs-node: Cache IRFetcher 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-30 14:22:12 +03:00
Alex Vanin
be6ae3c066 [#1278] neofs-node: Use global cached netmap source in services 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-30 14:22:12 +03:00
Alex Vanin
7ed84d1755 [#1278] acl: Return netmap.Source interface 
Application can provide cached netmap source in this case.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-30 14:22:12 +03:00
Evgenii Stratonikov
cf119e4ca9 [#1163] services/audit: Randomize the order of PDP checks 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-30 10:56:33 +03:00
Evgenii Stratonikov
dd9bd05bac [#1239] neofs-cli: Use pointer-less slices for object ID 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-29 16:02:05 +03:00
Evgenii Stratonikov
882236a03b [#1239] morph/client: Remove intermediate conversion in morph client 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-29 16:02:05 +03:00
Evgenii Stratonikov
6936195afa [#1239] util/attributes: Remove excessive slice copy during parsing 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-29 16:02:05 +03:00
Evgenii Stratonikov
2ad8016d75 [#1239] innerring: Use pointer-less slices for object IDs 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-29 16:02:05 +03:00
Pavel Karpy
6ec104d686 [#1255] node/session: Rename constant 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
90a8c52bdb [#1255] object: Add persistent storage usage 
Use persistent storage usage in the node if it was configured so.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
9cda3121ab [#1255] node/config: Add persistent storage 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
016eaa25f3 [#1255] node/session: Add encryption tests 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
01ed366e99 [#1255] node/session: Add encryption 
Add `WithEncryption` option that passes ECDSA key to the persistent session
storage. It uses 32 bytes from marshalled ECDSA key in ASN.1 DER from in
AES-256 algorithm encryption in Galois/Counter Mode.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
a884ad56d9 [#1255] node/session: Add persistent tests 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
455b9fb325 [#1255] node/session: Add persistent session storage 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Pavel Karpy
929c9851a6 [#1255] node/session: Create separate dir for in-memory storage 
Move in-memory session storage to the separate directory of `storage`. It is
done for future support of different kind of session storages.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-03-29 09:35:10 +03:00
Evgenii Stratonikov
2a69aaf976 [#1157] network/cache: Optimize client fetch from multiClient 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 18:11:20 +03:00
Evgenii Stratonikov
a4261243fc [#1157] network/cache: Cache multiclients based on public key only 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 18:11:20 +03:00
Evgenii Stratonikov
de5a2f6574 [#1262] metabase: Remove list index in place Delete 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 17:08:11 +03:00
Evgenii Stratonikov
43867a3093 [#1262] metabase: Do not allocate intermediate slices for indices 
```
name              old alloc/op   new alloc/op   delta
Put/parallel-8       123kB ± 4%     119kB ± 3%  -2.72%  (p=0.006 n=10+9)
Put/sequential-8     170kB ± 1%     168kB ± 1%  -1.42%  (p=0.000 n=10+10)

name              old allocs/op  new allocs/op  delta
Put/parallel-8         473 ± 1%       469 ± 0%  -0.87%  (p=0.000 n=10+10)
Put/sequential-8       792 ± 0%       787 ± 0%  -0.58%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 17:08:11 +03:00
Evgenii Stratonikov
0e9b6be3fd [#1262] metabase: Remove intermediate allocations in decodeList 
```
name              old time/op    new time/op    delta
Put/parallel-8      1.57ms ±11%    1.51ms ± 3%   -4.06%  (p=0.043 n=9+10)
Put/sequential-8    5.16ms ± 2%    5.16ms ± 3%     ~     (p=1.000 n=9+10)

name              old alloc/op   new alloc/op   delta
Put/parallel-8       126kB ± 4%     123kB ± 4%   -2.54%  (p=0.016 n=8+10)
Put/sequential-8     171kB ± 1%     170kB ± 1%     ~     (p=0.182 n=9+10)

name              old allocs/op  new allocs/op  delta
Put/parallel-8         565 ± 2%       473 ± 1%  -16.18%  (p=0.000 n=9+10)
Put/sequential-8       819 ± 1%       792 ± 0%   -3.34%  (p=0.000 n=9+10)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 17:08:11 +03:00
Evgenii Stratonikov
d45df614fb [#1262] metabase: Optimize decodeList 
Prevent additional allocation during `append` in `Put`.

```
name              old alloc/op   new alloc/op   delta
Put/parallel-8       131kB ± 1%     126kB ± 4%  -3.87%  (p=0.005 n=8+8)
Put/sequential-8     172kB ± 1%     171kB ± 1%  -0.73%  (p=0.028 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 17:08:11 +03:00
Evgenii Stratonikov
456e1584d6 [#1262] metabase: Add benchmarks for Put 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 17:08:11 +03:00
Evgenii Stratonikov
f1223b46df [#1262] blobovnicza: Make helper in Put function idempotent 
`Batch` can execute the function multiple times leading to multiple
increases of a size approximation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-28 17:08:11 +03:00
Alex Vanin
35ad6f188e Backport release v0.27.6 changelog 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-28 15:16:54 +03:00
Evgenii Stratonikov
571ae843ad [#1198] neofs-cli: Use io.Copy instead of io.CopyBuffer 
The buffer size value is somewhat arbitrary and making in configurable
doesn't make much sense, given that we can't really restrict total resource
consumption in other places. `Copy` uses 32 KiB buffer by default, which
is not big. This approach is also more flexible as we can get rid of
buffer completely by implementing `ReaderFrom`, `WriterTo` interfaces.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-25 18:21:48 +03:00
Evgenii Stratonikov
cbe07120da [#1261] neofs-cli: Allow to use relative epoch for bearer token 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-25 17:58:53 +03:00
Evgenii Stratonikov
f2c1bc4bfb [#1261] neofs-cli: Allow to create eACL with empty container ID 
Empty CID can be used in bearer token eACL.
See https://github.com/nspcc-dev/neofs-api/issues/207 .

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-25 17:58:53 +03:00
Evgenii Stratonikov
9b2523a408 [#1261] neofs-cli: Allow to create bearer tokens 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-25 17:58:53 +03:00
Evgenii Stratonikov
ae8e38cace [#1261] neofs-cli: Fix help message for acl extended create 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-25 17:58:53 +03:00
Alex Vanin
683439970a [#1270] neofs-node: Add timeout for grpc GracefulStop() 
GracefulStop() may be blocked until all server-side streams
are finished. There is no control over such streams yet, so
application may be frozen in shutdown stage.

Naive solution is to add timeout for GracefulStop(). At this
point healthy connection will be finished and unhealthy
connections will be terminated by Stop().

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-25 17:39:24 +03:00
Evgenii Stratonikov
ad92493b86 [#1268] blobstor: Cleanup zstd encoders/decoders 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-25 14:05:06 +03:00
Evgenii Stratonikov
4253931699 Release candidate v0.28.0-rc.2 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-24 17:51:59 +03:00