fyrchik/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
2653 commits 17 branches 26 tags 22 MiB
Commit graph

2653 commits

This branch
This branch
All branches
Author SHA1 Message Date
Evgenii Stratonikov
43e776dfb1 [#1462] writecache: Persist objects in batches 
`Update` becomes a botleneck under a heavy load.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
6ad87e7959 [#1462] writecache: Fill flush marks during startup 
Some of the objects are already flushed, don't do it twice.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
f2a7503964 [#1462] fstree: Allow to fetch file content lazily 
If we should process address based on some condition, there is no need
to read file content in memory.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
54d4503701 [#1462] writecache: Rename flushWorkersCount 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
0148209168 [#1462] writecache: Use Timer instead of Ticker in flush 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
5073a37930 [#1433] metabase: Optimize Select 
For some filters we can scan only a subset of keys instead of checking
each key.

```
name                       old time/op  new time/op  delta
Select/string_equal-8      49.3µs ± 4%  11.0µs ± 4%  -77.68%  (p=0.000 n=10+10)
Select/string_not_equal-8  7.01ms ± 5%  7.06ms ±10%     ~     (p=0.971 n=10+10)
Select/common_prefix-8      118µs ± 6%    79µs ± 5%  -33.04%  (p=0.000 n=10+9)
Select/unknown-8           21.3µs ± 4%   3.2µs ± 4%  -84.88%  (p=0.000 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
5c48588c64 [#1433] metabase: Add more tests for Select 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
ea76c989ca [#1433] metabase: Add a benchmark for Select 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
b602fff01f [#1463] blobovnicza: Remove pointers from struct members 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 13:42:54 +03:00
Evgenii Stratonikov
d9f0ac8909 [#1463] blobovnicza: Determine db size via os.Stat 
Currently we use `(*bbolt.Bucket).Stats().KeyN` for estimating database
size. However, it iterates over all pages in bucket and thus heavily
depends on the bucket size. This commit replaces initial size estimation
with a single `os.Stat` call.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 13:42:54 +03:00
Pavel Karpy
021aa97965 [#1418] blobovnicza: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
281befec67 [#1418] blobstor: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
babd382ba5 [#1418] engine: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
6e752f36dc [#1418] shard: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
e265ce2d52 [#1418] meta: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
1c100fb4b0 [#1418] writecache: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
da3ae202f0 [#1436] node: Log service initialization/boot up 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-02 16:16:39 +03:00
Evgenii Stratonikov
d84d52924a [#1467] pkg/util: Allow to load default account from a wallet 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-02 16:13:20 +03:00
Leonard Lyubich
2da8396a9f [#1454] cli/util: Support container sessions in sign session-token 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
ff576d125d [#1454] container: Fix reading session token from request origin 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
cc6209e8a0 [#1449] object/delete: Fix NPE in collectChain 
`head.previous` method returns (nil, nil) if object has no previous
neighbor.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-30 17:59:16 +03:00
Leonard Lyubich
96cdc04705 [#1449] policer: Unwrap status HEAD response 
Helper function `client.IsErrObjectNotFound` doesn't support error
unwrapping, so we need to do it on caller side.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-30 17:59:16 +03:00
Leonard Lyubich
e96ea4635c [#1449] policer: Fix selection of new storage candidates 
`Policer` should pass list of selected candidates into `WithNodes`
method of `replicator.Task`. In previous implementation `processNodes`
method passed an opposite list: failed nodes and/or the local one.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-30 17:59:16 +03:00
Evgenii Stratonikov
855cbf5a3a [#1159] services/container: Remove ContainerWithContext struct 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-30 15:40:22 +03:00
Leonard Lyubich
2bcc0051ab [#1423] session: Get session issuer from token structure 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
2dec3a0a93 [#1423] ir/container: Fix error message related to container removal 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
883f2f1ac6 [#1423] local storage: Activate skipped unit tests after fix 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
4c8ec20e32 [#1423] session: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Evgenii Stratonikov
dda56f1319 [#1437] neofs-cli: Move util command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-26 12:03:39 +03:00
Pavel Karpy
786c920fb2 [#1430] go.mod: Update neo-go 
v0.98.2 => v0.98.3

That patch includes fixing panic when using WS client with a closed
connection.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-25 13:54:58 +03:00
Pavel Karpy
57c5fccc8c [#1428] node/acl: Make OID optional 
Not all the NeoFS requests must contain OID in their bodies (or must NOT
contain them at all). Do not pass object address in helper functions, pass
CID and OID separately instead.
Also, fixed NPE in the ACL service: updated SDK library brought errors
when working with `Put` and `Search` requests without OID fields.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-25 12:11:03 +03:00
Pavel Karpy
d69eb2aaf3 [#1428] node/acl: Make container ID as required param 
Change pointer to value in request information since requests could not
exist without container ID.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-25 12:11:03 +03:00
Evgenii Stratonikov
6cb9c13c5e [#1424] neofs-cli: Fail immediately if a client can't be created 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
295ec3700a [#1424] neofs-cli: Fail immediately if a key can't be fetched 
If the key can't be fetched, an error is always returned, so it makes
sense to fail the whole command inside of a `key.Get*()`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
d9c5ca5e77 [#1381] neofs-cli: Move control command to a separate module 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
71d823f192 [#1381] neofs-cli: Move JSON and verbose printing to a common package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-25 09:46:41 +03:00
Evgenii Stratonikov
c57114def3 [#1422] morph/client: Fetch group key properly 
NNS contract `resolve` method returns an array.
`neofs-adm` is already up to date.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-23 19:24:46 +03:00
Leonard Lyubich
86552cf3ae engine: make corruptSubDir to ignore blobovnicza dir 
There is a dir for Blobovnicza tree in BlobStor, so we need to ignore
it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:33:14 +03:00
Leonard Lyubich
bb25ecbd15 [#1400] owner: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:33:14 +03:00
Leonard Lyubich
f8ac4632f8 [#1335] policer: Prevent potential object loss 
In previous implementation `Policer` considered local object copy as
redundant on processing single placement vector.

Make `Policer` to call redundant copy callback after full placement
processing. Also fix 404 error parsing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:24:23 +03:00
Pavel Karpy
9f62d25b50 [#1224] ir: Delete nodes that do not have subnets 
Also, update their subnet entrance on subnet removal events.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
b3e1e90c01 [#1224] ir: Listen to subnet removal events 
IR must listen to subnet removal notifications always (regardless of notary
mode).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
71f18ba9ec [#1224] morph: Call Alphabet methods flexibly 
Call `UpdateStateIR` and `AddPeerIR` method instead of `UpdateState` and
`AddPeer` if calling client is configured as Alphabet in notary enabled
environment.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
a129fc98da [#1224] morph: Add configuration getters 
Includes `WithNotary` and `IsAlpha` methods to the `StaticClient`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
f0c9a68ad3 [#1224] morph: Delete wrong comment 
Current implementation does not use MurmurHash3.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Evgenii Stratonikov
df48ddcb3d Revert "[#1141] morph/client: Allow to use more integer types as arguments" 
This reverts commit 9349f422fd9a29ec529ef0648bbfabd1f9ff7651.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 16:45:15 +03:00
Evgenii Stratonikov
4761857fb3 [#1396] cmd: Allow to autogenerate documentation for all commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:35:22 +03:00
Evgenii Stratonikov
c6325fdc91 [#1379] neofs-cli: Move accounting command to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00
Evgenii Stratonikov
56f33436dd [#1379] neofs-cli: Move exitOnErr to internal package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:20:34 +03:00