Commit graph

1688 commits

Author SHA1 Message Date
Pavel Karpy
b3e1e90c01 [#1224] ir: Listen to subnet removal events
IR must listen to subnet removal notifications always (regardless of notary
mode).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy
71f18ba9ec [#1224] morph: Call Alphabet methods flexibly
Call `UpdateStateIR` and `AddPeerIR` method instead of `UpdateState` and
`AddPeer` if calling client is configured as Alphabet in notary enabled
environment.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy
a129fc98da [#1224] morph: Add configuration getters
Includes `WithNotary` and `IsAlpha` methods to the `StaticClient`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy
f0c9a68ad3 [#1224] morph: Delete wrong comment
Current implementation does not use MurmurHash3.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Evgenii Stratonikov
df48ddcb3d Revert "[#1141] morph/client: Allow to use more integer types as arguments"
This reverts commit 9349f422fd9a29ec529ef0648bbfabd1f9ff7651.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 16:45:15 +03:00
Evgenii Stratonikov
4761857fb3 [#1396] cmd: Allow to autogenerate documentation for all commands
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 13:35:22 +03:00
Pavel Karpy
4be5dce848 [#1405] ir: Fix settlement log message
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-20 12:40:12 +03:00
Evgenii Stratonikov
3f2889a4e9 [#1392] eacl/v2: Use EncodeToString where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
63cdc16088 [#1386] eacl/v2: Do not allocate intermediate slices for XHeaders
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
915f87959e [#1386] eacl/v2: Allocate []eaclSDK.Header slice immediately
There is no point in having intermediate `[]sessionSDK.XHeader`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
b24589b62d [#1386] eacl/v2: Use raw structs where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov
3f4475f97b [#1386] services/object: Fail eACL check if the request is invalid
Parse all headers beforehand and reject invalid requests.
Another approach would be to remember the error and check
it after `CalculateAction`, which is a bit faster.
The rule of thumb here is "first validate, then use".

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Pavel Karpy
f99a0498da [#1403] util: Add details to errors
Make error messages more descriptive when parsing LOCODE from string errors
appear.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-20 10:53:27 +03:00
Leonard Lyubich
aeb9884218 [#1389] crypto: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-19 10:12:14 +03:00
Leonard Lyubich
5e50ddd7f5 [#1389] storagegroup: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-19 10:12:14 +03:00
Evgenii Stratonikov
0f376a5d83 [#1388] services/control: Extend tests for ShardInfo marshaling
It is nice to have different paths for different components and also
check that the information returned is different for different shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-17 13:48:07 +03:00
Leonard Lyubich
f15e6e888f [#1377] oid, cid: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-16 15:33:22 +03:00
Evgenii Stratonikov
f65898a354 [#1141] morph/client: Use boolean type directly
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-16 15:09:35 +03:00
Evgenii Stratonikov
230a5cd037 [#1141] morph/client: Remove unneeded argument conversions
Client supports `util.Uint160` directly, no need to convert it to bytes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-16 15:09:35 +03:00
Evgenii Stratonikov
1e8391d216 [#1141] morph/client: Allow to use more integer types as arguments
Also, use `*big.Int` as integer value, see nspcc-dev/neo-go#2413.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-16 15:09:35 +03:00
Leonard Lyubich
3a188bb2e5 [#1371] bearer: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 15:16:17 +03:00
Leonard Lyubich
ae92074272 [#1369] checksum: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
Leonard Lyubich
088df0e2a9 [#1369] version: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
Leonard Lyubich
7a57d7b076 [#1369] audit: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
Elizaveta Chichindaeva
cc7a723d77 [#1320] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-11 10:40:02 +03:00
Leonard Lyubich
cd545f0160 [#1351] cli: Fix connection scheme loss during endpoint parsing
In previous implementation NeoFS CLI app used `network.Address.HostAddr`
as a server URI, which caused scheme loss since host address doesn't
contain it.

Rename `HostAddr` to `URIAddr` and make it to return URI address with
`grpcs` scheme if TLS is enabled. Make `TLSEnabled` unexported since it
was used to provide default `tls.Config` only (it is used by default in
SDK).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-05 10:43:27 +03:00
Evgenii Stratonikov
a4769d8624 *: use require.ErrorIs where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:47:57 +03:00
Evgenii Stratonikov
3c39e6df11 [#1319] blobstor: Compress big objects in a streaming fashion
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Evgenii Stratonikov
1219ff89d4 [#1319] services/object_manager: Fix error message
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Evgenii Stratonikov
057d53459b [#1319] services/object: Remove Write method from local and remote targets
They work with prepared objects only.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Evgenii Stratonikov
4ea03c01b5 [#1319] services/object: Store payload directly in distributedTarget
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Pavel Karpy
9da7df4d42 [#1318] metabase: Init metabase in test
`Init()` creates buckets. It prevents NPE in tests.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
486ec8a2e5 [#1318] metabase: Reuse comparing func from require
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
1caeb0b648 [#1318] metabase: Add missing error check in tests
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
6f2723b47e [#1318] node: Fix linter
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
2aec5736e0 [#1318] node/gc: Fix tombstones handling
Handle only expired tombstones.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
24ca8ca5c7 [#1318] object: Change default ttl
This is need to reuse object service without passing any API v2 structs.
Also, fix comment.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
7799f8e4cf [#1318] engine: Change tombstone clear process
- Delete objects physically on tombstone's arrival;
- Store information about tombstones in the Graveyard;
- Clear Graveyard every epoch based on the information about TS in the
network.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
e4cfeec449 [#1318] service: Add tombstone checker service
The service fetches tombstones from the network via object service, every
request is handled in the following order:
1. checks local LRU cache;
2. checks local storage engine;
3. tries to find object in the placement nodes.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
2583f608e8 [#1318] metabase: Add DropGraves method
Method clean up Graveyard and should be called by the routine that handles
tombstone expiration.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
daab30c391 [#1318] metabase: Add iteration offset
Add offset element to the iterations over deleted objects (both the
Graveyard and the Garbage buckets).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
fe8076e60a [#1318] metabase: Add immediate object deletion
Mark objects with GC immediately after a Tombstone is received.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy
8107c8d1a9 [#1318] metabase: Separate buckets with TS and GC marks
It allows storing information about object in both ways at the same time:
1. Metabase should know if an object is covered by a tombstone (that is
not expired yet);
2. It should be possible to physically delete objects covered by a
tombstone immediately (mark with GC) but keep tombstone knowledge.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Evgenii Stratonikov
5736b834c3 [#1286] services/object: Validate object when created with sesssion token
All fields set by a user should be verified.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-25 10:53:22 +03:00
Evgenii Stratonikov
c6a9c5cd8c [#1283] services/object: Disallow creating objects without a session token
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-25 10:53:22 +03:00
Pavel Karpy
bd27837364 [#1321] node: Register GC event channel before shard init
Morph "NewEpoch" event handling was registered in a closure over
`addNewEpochNotificationHandler` func. That may lead to the data race:
if a shard was initialized before the event registration, everything works
as planned, but if registration was made earlier, it was not able to
include GC handlers since a shard has not called `eventChanInit` yet and,
therefore, it has not registered handler yet.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-25 10:39:37 +03:00
Evgenii Stratonikov
6c08ec9ca2 [#1312] morph/netmap: Fix panic during parameter parsing
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-12 17:37:12 +03:00
Pavel Karpy
b6e5aac604 [#1170] pkg/morph: Delete unused function
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-08 11:52:18 +03:00
Pavel Karpy
01e69f2f7a [#1170] pkg/morph: Add worker pool
Add worker pool to the listener to prevent blocking. It is used only for
notary notifications and new block events handling since it uses RPC
calls. That may lead to the deadlock state: neo-go cannot send RPC until
notification channel is read but notification channel cannot be read since
neo-go client cannot send RPC.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-08 11:52:18 +03:00
Pavel Karpy
4ed5b1ceef [#1170] pkg/morph: Invalidate cache on RPC switch
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-08 11:52:18 +03:00