acid-ant/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
2552 commits 108 branches 26 tags 28 MiB
Commit graph

1721 commits

Author SHA1 Message Date
Evgenii Stratonikov
6ad87e7959 [#1462] writecache: Fill flush marks during startup 
Some of the objects are already flushed, don't do it twice.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
f2a7503964 [#1462] fstree: Allow to fetch file content lazily 
If we should process address based on some condition, there is no need
to read file content in memory.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
54d4503701 [#1462] writecache: Rename flushWorkersCount 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
0148209168 [#1462] writecache: Use Timer instead of Ticker in flush 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
5073a37930 [#1433] metabase: Optimize Select 
For some filters we can scan only a subset of keys instead of checking
each key.

```
name                       old time/op  new time/op  delta
Select/string_equal-8      49.3µs ± 4%  11.0µs ± 4%  -77.68%  (p=0.000 n=10+10)
Select/string_not_equal-8  7.01ms ± 5%  7.06ms ±10%     ~     (p=0.971 n=10+10)
Select/common_prefix-8      118µs ± 6%    79µs ± 5%  -33.04%  (p=0.000 n=10+9)
Select/unknown-8           21.3µs ± 4%   3.2µs ± 4%  -84.88%  (p=0.000 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
5c48588c64 [#1433] metabase: Add more tests for Select 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
ea76c989ca [#1433] metabase: Add a benchmark for Select 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
b602fff01f [#1463] blobovnicza: Remove pointers from struct members 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 13:42:54 +03:00
Evgenii Stratonikov
d9f0ac8909 [#1463] blobovnicza: Determine db size via os.Stat 
Currently we use `(*bbolt.Bucket).Stats().KeyN` for estimating database
size. However, it iterates over all pages in bucket and thus heavily
depends on the bucket size. This commit replaces initial size estimation
with a single `os.Stat` call.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 13:42:54 +03:00
Pavel Karpy
021aa97965 [#1418] blobovnicza: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
281befec67 [#1418] blobstor: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
babd382ba5 [#1418] engine: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
6e752f36dc [#1418] shard: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
e265ce2d52 [#1418] meta: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
1c100fb4b0 [#1418] writecache: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Evgenii Stratonikov
d84d52924a [#1467] pkg/util: Allow to load default account from a wallet 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-02 16:13:20 +03:00
Leonard Lyubich
ff576d125d [#1454] container: Fix reading session token from request origin 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
cc6209e8a0 [#1449] object/delete: Fix NPE in collectChain 
`head.previous` method returns (nil, nil) if object has no previous
neighbor.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-30 17:59:16 +03:00
Leonard Lyubich
96cdc04705 [#1449] policer: Unwrap status HEAD response 
Helper function `client.IsErrObjectNotFound` doesn't support error
unwrapping, so we need to do it on caller side.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-30 17:59:16 +03:00
Leonard Lyubich
e96ea4635c [#1449] policer: Fix selection of new storage candidates 
`Policer` should pass list of selected candidates into `WithNodes`
method of `replicator.Task`. In previous implementation `processNodes`
method passed an opposite list: failed nodes and/or the local one.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-30 17:59:16 +03:00
Evgenii Stratonikov
855cbf5a3a [#1159] services/container: Remove ContainerWithContext struct 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-30 15:40:22 +03:00
Leonard Lyubich
2bcc0051ab [#1423] session: Get session issuer from token structure 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
2dec3a0a93 [#1423] ir/container: Fix error message related to container removal 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
883f2f1ac6 [#1423] local storage: Activate skipped unit tests after fix 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
4c8ec20e32 [#1423] session: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Pavel Karpy
57c5fccc8c [#1428] node/acl: Make OID optional 
Not all the NeoFS requests must contain OID in their bodies (or must NOT
contain them at all). Do not pass object address in helper functions, pass
CID and OID separately instead.
Also, fixed NPE in the ACL service: updated SDK library brought errors
when working with `Put` and `Search` requests without OID fields.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-25 12:11:03 +03:00
Pavel Karpy
d69eb2aaf3 [#1428] node/acl: Make container ID as required param 
Change pointer to value in request information since requests could not
exist without container ID.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-25 12:11:03 +03:00
Evgenii Stratonikov
c57114def3 [#1422] morph/client: Fetch group key properly 
NNS contract `resolve` method returns an array.
`neofs-adm` is already up to date.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-23 19:24:46 +03:00
Leonard Lyubich
86552cf3ae engine: make corruptSubDir to ignore blobovnicza dir 
There is a dir for Blobovnicza tree in BlobStor, so we need to ignore
it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:33:14 +03:00
Leonard Lyubich
bb25ecbd15 [#1400] owner: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:33:14 +03:00
Leonard Lyubich
f8ac4632f8 [#1335] policer: Prevent potential object loss 
In previous implementation `Policer` considered local object copy as
redundant on processing single placement vector.

Make `Policer` to call redundant copy callback after full placement
processing. Also fix 404 error parsing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:24:23 +03:00
Pavel Karpy
9f62d25b50 [#1224] ir: Delete nodes that do not have subnets 
Also, update their subnet entrance on subnet removal events.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
b3e1e90c01 [#1224] ir: Listen to subnet removal events 
IR must listen to subnet removal notifications always (regardless of notary
mode).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
71f18ba9ec [#1224] morph: Call Alphabet methods flexibly 
Call `UpdateStateIR` and `AddPeerIR` method instead of `UpdateState` and
`AddPeer` if calling client is configured as Alphabet in notary enabled
environment.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
a129fc98da [#1224] morph: Add configuration getters 
Includes `WithNotary` and `IsAlpha` methods to the `StaticClient`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Pavel Karpy
f0c9a68ad3 [#1224] morph: Delete wrong comment 
Current implementation does not use MurmurHash3.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-23 08:16:24 +03:00
Evgenii Stratonikov
df48ddcb3d Revert "[#1141] morph/client: Allow to use more integer types as arguments" 
This reverts commit 9349f422fd9a29ec529ef0648bbfabd1f9ff7651.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 16:45:15 +03:00
Evgenii Stratonikov
4761857fb3 [#1396] cmd: Allow to autogenerate documentation for all commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 13:35:22 +03:00
Pavel Karpy
4be5dce848 [#1405] ir: Fix settlement log message 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-20 12:40:12 +03:00
Evgenii Stratonikov
3f2889a4e9 [#1392] eacl/v2: Use EncodeToString where possible 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
63cdc16088 [#1386] eacl/v2: Do not allocate intermediate slices for XHeaders 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
915f87959e [#1386] eacl/v2: Allocate []eaclSDK.Header slice immediately 
There is no point in having intermediate `[]sessionSDK.XHeader`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
b24589b62d [#1386] eacl/v2: Use raw structs where possible 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
3f4475f97b [#1386] services/object: Fail eACL check if the request is invalid 
Parse all headers beforehand and reject invalid requests.
Another approach would be to remember the error and check
it after `CalculateAction`, which is a bit faster.
The rule of thumb here is "first validate, then use".

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-20 12:04:10 +03:00
Pavel Karpy
f99a0498da [#1403] util: Add details to errors 
Make error messages more descriptive when parsing LOCODE from string errors
appear.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-05-20 10:53:27 +03:00
Leonard Lyubich
aeb9884218 [#1389] crypto: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-19 10:12:14 +03:00
Leonard Lyubich
5e50ddd7f5 [#1389] storagegroup: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-19 10:12:14 +03:00
Evgenii Stratonikov
0f376a5d83 [#1388] services/control: Extend tests for ShardInfo marshaling 
It is nice to have different paths for different components and also
check that the information returned is different for different shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-05-17 13:48:07 +03:00
Leonard Lyubich
f15e6e888f [#1377] oid, cid: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-16 15:33:22 +03:00